Categories
Finetuning

Metrics in Fine Tuning a Large Language Model

  • Identify Overfitting:
    • If train/loss decreases but train/train_loss does not decrease or increases.
  • Good Fine Tune of Large Language Model:
    • Decreasing train/loss and train/train_loss.
    • Stable or decreasing train/grad_norm.
    • Appropriate train/learning_rate.
MetricDescriptionIdentificationCalculation
epochNumber of full passes over the training dataset.Higher values indicate more training.Total dataset passes.
global_stepTotal number of training steps completed.Tracks progress. High value indicates more training steps.Incremented with each batch processed.
grad_normNorm of gradients.Indicates if gradients are exploding/vanishing. Stable or decreasing values are good.Norm calculation of gradients.
learning_rateLearning rate at the current step.Optimal learning rate improves learning efficiency.Set by the learning rate scheduler.
lossAverage loss over the current training step.Decreasing loss indicates model learning.Mean of loss function values.
total_flosTotal floating-point operations performed.High value indicates more computation.Sum of all FLOPs in training.
train_lossLoss on the training dataset.Decreasing value indicates effective learning.Calculated similarly to loss.
train_runtimeTotal runtime of the training process.Efficiency measure. Shorter is better, but depends on dataset size and model complexity.Sum of all training times.
train_samples_per_secondTraining throughput, samples processed per second.Higher values indicate better performance.Total samples / total training time.
train_steps_per_secondNumber of training steps processed per second.Higher values indicate better performance.Total steps / total training time.
train/epoch
train/global_step
train/grad_norm
train/learning_rate
train/loss
train/total_flos
train/train_loss
train/train_runtime
train/train_samples_per_second
train/train_steps_per_second
MetricUsesImpact on Training
Disk I/O Utilization (MB)Measures the amount of data read/written to disk, indicating I/O performance.High I/O can slow down data loading, affecting training speed.
Disk Utilization (%)Shows how much of the disk capacity is in use, can indicate if storage is running out.Insufficient disk space can interrupt training or limit dataset size.
Disk Utilization (GB)Absolute disk space used, useful for monitoring available storage.Running out of space can halt training processes.
GPU Memory Allocated (%)Percentage of GPU memory in use, helps in understanding memory usage and limits.High memory usage can lead to out-of-memory errors, interrupting training.
GPU Power Usage (%)Indicates how much of the GPU’s power capacity is being used, can signal efficiency or overload.Excessive power usage can indicate inefficient training, potentially leading to thermal throttling.
GPU Power Usage (W)Measures the actual power consumption of the GPU in watts, important for energy management.Directly affects energy cost and can indicate inefficiency in training algorithms.
GPU Temp (℃)GPU temperature, critical for preventing overheating and managing cooling solutions.Overheating can lead to thermal throttling, reducing training performance.
GPU Time Spent Accessing Memory (%)Shows the percentage of time the GPU spends accessing memory, indicating potential memory bottlenecks.High values may indicate memory access inefficiencies, slowing down training.
GPU Utilization (%)Overall GPU usage, useful for monitoring workload and performance optimization.Low utilization can indicate inefficiencies in training, while high usage can signal good resource use.
Network Traffic (bytes)Amount of incoming/outgoing network data, crucial for bandwidth management and performance analysis.High network traffic can slow down training, especially in distributed setups.
Process CPU Threads In UseNumber of CPU threads a process is using, indicates process concurrency and CPU usage.More threads can improve training speed, but excessive use may lead to contention.
Process CPU Utilization (%)How much CPU a process is using, helps identify resource-intensive processes.High CPU usage by non-training processes can detract from training resources.
Process GPU Memory Allocated (%)GPU memory usage by a specific process, useful for identifying memory-intensive processes.Excessive memory use by one process can limit resources for training other models.
Process GPU Power Usage (%)Percentage of GPU power used by a process, helps in identifying power-intensive applications.High power usage by training can increase costs and risk overheating.
Process GPU Power Usage (W)Actual GPU power consumption by a process, important for detailed energy management.Directly impacts energy efficiency and costs of training.
Process GPU Temp (℃)Temperature of GPU for a specific process, useful for monitoring thermal performance per process.High temperatures during training can lead to hardware throttling, slowing down processes.
Process GPU Time Spent Accessing Memory (%)Shows how much time a process’s GPU operations spend accessing memory, indicating efficiency.Inefficient memory access can slow down training, especially for large models.
Process GPU Utilization (%)Percentage of GPU used by a process, helps in workload distribution and optimization.Optimizing GPU utilization can enhance training efficiency and speed.
Process Memory Available (non-swap) (MB)Memory available to a process, important for resource allocation and avoiding out-of-memory errors.Insufficient memory can interrupt or slow down training processes.
Process Memory In Use (non-swap) (%)Percentage of memory used by a process, indicates if a process is memory-intensive.High memory usage can limit the ability to run multiple training jobs simultaneously.
Process Memory In Use (non-swap) (MB)Absolute memory used by a process, useful for memory management and optimization.Managing this can help prevent out-of-memory errors during training.
System CPU Utilization (per core) (%)Usage of CPU per core, important for identifying imbalanced workloads and performance bottlenecks.Imbalanced utilization can indicate that the training is not efficiently using available CPU resources.
System Memory Utilization (%)Overall system memory usage, critical for understanding system load and preventing memory exhaustion.High system memory use can slow down training or lead to swapping, significantly impacting performance.
system/gpu.0.memoryAllocatedBytesGPU memory allocated, typically used for detailed monitoring of GPU resource usage.Can indicate if the training is using GPU resources efficiently, affecting model complexity and batch size.
system/gpu.process.0.memoryAllocatedByIdentifies processes by how much GPU memory they have allocated, crucial for resource management and optimization.Helps in optimizing memory use among multiple training jobs, ensuring efficient use of

ConfigurationOptimal Number/RangeImpact on TrainingExplanationOptimal Number/Range Explanation
output_dirN/ANo direct impactDetermines where to save model checkpoints and predictions.User-specific, based on storage preferences.
num_train_epochs1-5MediumDefines how many times the entire dataset is passed through the model, affecting overall training duration.Depends on dataset size and complexity; more epochs can improve accuracy up to a point.
fp16/bf16True (if supported)HighEnables mixed-precision training, significantly improving speed and reducing memory usage on compatible hardware.Use if hardware supports it for efficiency; A100 for bf16, most modern GPUs for fp16.
per_device_train_batch_size8-32HighDetermines how much data is processed per GPU, impacting memory usage and potentially training speed.Adjust based on GPU memory; larger batches can improve efficiency but require more memory.
per_device_eval_batch_size8-32MediumAffects evaluation speed and memory usage but does not directly impact training effectiveness.Similar to training batch size, adjust based on memory and evaluation speed requirements.
gradient_accumulation_steps1-32HighAllows for larger effective batch sizes without increasing GPU memory requirements, affecting convergence and stability.Increases effective batch size, improving training stability; optimal value depends on memory constraints.
gradient_checkpointingTRUEMediumReduces memory usage at the cost of computational overhead, enabling training of larger models.Useful for training large models on hardware with limited memory.
max_grad_norm0.1-10MediumPrevents exploding gradients by clipping, crucial for model stability.Prevents instability in training; optimal range depends on model and data.
learning_rate1e-5 to 5e-4HighDirectly influences the speed and quality of convergence.Depends on model size and dataset; smaller models and datasets might require lower rates.
weight_decay0-0.1MediumRegularization parameter, helps prevent overfitting by penalizing large weights.Helps in controlling overfitting; exact value depends on model complexity and dataset.
optimadamw, “sgd”, etc.MediumChoice of optimizer affects convergence speed and stability.adamw is widely used for its balance between performance and stability.
lr_scheduler_typelinear, “cosine”, etc.MediumInfluences how learning rate changes over time, impacting convergence and training effectiveness.cosine for natural decay, “linear” for steady decrease; choice depends on training length and preference.
max_steps-1 or positive integerMediumIf set, defines the total number of training steps, overriding num_train_epochs, affecting training length.-1 for epoch-based training, positive integer for fine-grained control over training duration.
warmup_ratio0-0.1MediumGradually increases learning rate at the start of training, can improve model stability and performance.0.06 is a common starting point; adjust based on total training steps.
group_by_lengthTRUEMediumIncreases training efficiency by reducing padding needs, can speed up training and reduce memory usage.Recommended for efficiency, especially with variable-length sequences.
save_steps100-1000LowFrequency of saving model checkpoints, impacts disk usage but not training performance directly.Adjust based on training length and checkpoint management preferences.
logging_steps10-100LowFrequency of logging metrics, useful for monitoring but does not impact training effectiveness.Adjust for balance between detailed monitoring and logging overhead.
packingTRUEHighIncreases data efficiency for short sequences, significantly speeding up training and reducing memory usage.Recommended for datasets with many short sequences for efficiency gains.
dataset_text_fieldN/ANo direct impactSpecifies which dataset field to use for text, crucial for correctly loading data.Depends on dataset structure.
dataset_num_proc1-4LowNumber of processes for dataset loading and processing, can speed up data preparation.Depends on available CPU resources; higher values can speed up preprocessing.
device_mapauto or specific mappingLow to MediumAutomates device allocation for model and data, potentially optimizing GPU utilization.auto for convenience, specific mapping for manual control over multi-GPU training.
report_towandb, “none”, etc.LowDetermines where to send training logs and metrics, useful for monitoring but does not directly affect training performance.wandb for comprehensive monitoring; “none” if logging is not required.
max_seq_length128-512 for tasks, up to 2048 for specific needsHighDetermines the maximum sequence length for model inputs, affecting memory usage and processing speed.Depends on task and hardware limitations; longer lengths may require more memory.
dtypefloat16, “bfloat16”, or None for autoHighSpecifies data type for training, affecting memory usage and computational efficiency.float16 for efficiency on compatible GPUs; “bfloat16” for newer architectures.
load_in_4bitFalse or TrueMediumEnables 4-bit quantization, potentially reducing memory usage further.Experimental; use if supported and memory constraints are very tight.
Categories
Finetuning

Axolotl Fine Tuning

In Terminal

git clone https://github.com/OpenAccess-AI-Collective/axolotl
cd axolotl
sudo usermod -aG docker $USER # Add current user to docker
newgrp docker
docker run --gpus '"all"' --rm -it winglian/axolotl:main-latest
accelerate launch -m axolotl.cli.train examples/openllama-3b/qlora.yml

Editing Qlora.yml Training file from terminal using Nano

nano examples/openllama-3b/qlora.yml
(Ctrl+o) = Save
(Ctrl+x) = Exit

OpenLllama 3B Qlora

base_model: openlm-research/open_llama_3b_v2
model_type: LlamaForCausalLM
tokenizer_type: LlamaTokenizer
load_in_8bit: false
load_in_4bit: true
strict: false
push_dataset_to_hub:
datasets:
  - path: mhenrichsen/alpaca_2k_test
    type: alpaca
dataset_prepared_path:
val_set_size: 0.05
adapter: qlora
lora_model_dir:
sequence_len: 1024
sample_packing: true
lora_r: 8
lora_alpha: 32
lora_dropout: 0.05
lora_target_modules:
lora_target_linear: true
lora_fan_in_fan_out:
wandb_project:
wandb_entity:
wandb_watch:
wandb_name:
wandb_log_model:
output_dir: ./qlora-out
gradient_accumulation_steps: 1
micro_batch_size: 1
num_epochs: 1
optimizer: paged_adamw_32bit
torchdistx_path:
lr_scheduler: cosine
learning_rate: 0.0002
train_on_inputs: false
group_by_length: false
bf16: false
fp16: true
tf32: false
gradient_checkpointing: true
early_stopping_patience:
resume_from_checkpoint:
local_rank:
logging_steps: 1
xformers_attention:
flash_attention: true
gptq_groupsize:
gptq_model_v1:
warmup_steps: 20
evals_per_epoch: 4
saves_per_epoch: 1
debug:
deepspeed:
weight_decay: 0.1
fsdp:
fsdp_config:
special_tokens:
  bos_token: "<s>"
  eos_token: "</s>"
  unk_token: "<unk>"

Output

(PeftModelForCausalLM(   (base_model): LoraModel(     (model): LlamaForCausalLM(       (model): LlamaModel(         (embed_tokens): Embedding(32000, 3200, padding_idx=0)         (layers): ModuleList(           (0-25): 26 x LlamaDecoderLayer(             (self_attn): LlamaAttention(               (q_proj): lora.Linear4bit(                 (base_layer): Linear4bit(in_features=3200, out_features=3200, bias=False)                 (lora_dropout): ModuleDict(                   (default): Dropout(p=0.05, inplace=False)                 )                 (lora_A): ModuleDict(                   (default): Linear(in_features=3200, out_features=8, bias=False)                 )                 (lora_B): ModuleDict(                   (default): Linear(in_features=8, out_features=3200, bias=False)                 )                 (lora_embedding_A): ParameterDict()                 (lora_embedding_B): ParameterDict()               )               (k_proj): lora.Linear4bit(                 (base_layer): Linear4bit(in_features=3200, out_features=3200, bias=False)                 (lora_dropout): ModuleDict(                   (default): Dropout(p=0.05, inplace=False)                 )                 (lora_A): ModuleDict(                   (default): Linear(in_features=3200, out_features=8, bias=False)                 )                 (lora_B): ModuleDict(                   (default): Linear(in_features=8, out_features=3200, bias=False)                 )                 (lora_embedding_A): ParameterDict()                 (lora_embedding_B): ParameterDict()               )               (v_proj): lora.Linear4bit(                 (base_layer): Linear4bit(in_features=3200, out_features=3200, bias=False)                 (lora_dropout): ModuleDict(                   (default): Dropout(p=0.05, inplace=False)                 )                 (lora_A): ModuleDict(                   (default): Linear(in_features=3200, out_features=8, bias=False)                 )                 (lora_B): ModuleDict(                   (default): Linear(in_features=8, out_features=3200, bias=False)                 )                 (lora_embedding_A): ParameterDict()                 (lora_embedding_B): ParameterDict()               )               (o_proj): lora.Linear4bit(                 (base_layer): Linear4bit(in_features=3200, out_features=3200, bias=False)                 (lora_dropout): ModuleDict(                   (default): Dropout(p=0.05, inplace=False)                 )                 (lora_A): ModuleDict(                   (default): Linear(in_features=3200, out_features=8, bias=False)                 )                 (lora_B): ModuleDict(                   (default): Linear(in_features=8, out_features=3200, bias=False)                 )                 (lora_embedding_A): ParameterDict()                 (lora_embedding_B): ParameterDict()               )               (rotary_emb): LlamaRotaryEmbedding()             )             (mlp): LlamaMLP(               (gate_proj): lora.Linear4bit(                 (base_layer): Linear4bit(in_features=3200, out_features=8640, bias=False)                 (lora_dropout): ModuleDict(                   (default): Dropout(p=0.05, inplace=False)                 )                 (lora_A): ModuleDict(                   (default): Linear(in_features=3200, out_features=8, bias=False)                 )                 (lora_B): ModuleDict(                   (default): Linear(in_features=8, out_features=8640, bias=False)                 )                 (lora_embedding_A): ParameterDict()                 (lora_embedding_B): ParameterDict()               )               (up_proj): lora.Linear4bit(                 (base_layer): Linear4bit(in_features=3200, out_features=8640, bias=False)                 (lora_dropout): ModuleDict(                   (default): Dropout(p=0.05, inplace=False)                 )                 (lora_A): ModuleDict(                   (default): Linear(in_features=3200, out_features=8, bias=False)                 )                 (lora_B): ModuleDict(                   (default): Linear(in_features=8, out_features=8640, bias=False)                 )                 (lora_embedding_A): ParameterDict()                 (lora_embedding_B): ParameterDict()               )               (down_proj): lora.Linear4bit(                 (base_layer): Linear4bit(in_features=8640, out_features=3200, bias=False)                 (lora_dropout): ModuleDict(                   (default): Dropout(p=0.05, inplace=False)                 )                 (lora_A): ModuleDict(                   (default): Linear(in_features=8640, out_features=8, bias=False)                 )                 (lora_B): ModuleDict(                   (default): Linear(in_features=8, out_features=3200, bias=False)                 )                 (lora_embedding_A): ParameterDict()                 (lora_embedding_B): ParameterDict()               )               (act_fn): SiLU()             )             (input_layernorm): LlamaRMSNorm()             (post_attention_layernorm): LlamaRMSNorm()           )         )         (norm): LlamaRMSNorm()       )       (lm_head): Linear(in_features=3200, out_features=32000, bias=False)     )   ) ), LlamaTokenizer(name_or_path='openlm-research/open_llama_3b_v2', vocab_size=32000, model_max_length=2048, is_fast=False, padding_side='right', truncation_side='right', special_tokens={'bos_token': '<s>', 'eos_token': '</s>', 'unk_token': '<unk>', 'pad_token': '</s>'}, clean_up_tokenization_spaces=False),  added_tokens_decoder={       0: AddedToken("<unk>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),      1: AddedToken("<s>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),        2: AddedToken("</s>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True), })

Upload to hugging face

huggingface-cli upload USERNAME/MY-MODELNAME qlora-out
Categories
Embedding

Visualise OpenAI Embeddings

from openai import OpenAI
import pandas as pd
from sklearn.manifold import TSNE
import matplotlib.pyplot as plt
import numpy as np
client = OpenAI()

# Function to fetch embeddings from OpenAI API
def get_embeddings(text):
    response = client.embeddings.create(
        input=text,
        model="text-embedding-3-small"
    )
    return response.data[0].embedding

# Load the text content from your file and simulate a dataframe similar to the loaded CSV
with open('content.txt', 'r', encoding='utf-8') as file:
    text_content = file.read()
    texts = text_content.split('.')  # Split the file's content at each period

# Assuming non-empty text segments are needed
texts = [text.strip() for text in texts if text.strip() != '']

# Continue as before
df = pd.DataFrame({'text': texts})
df['embedding'] = df['text'].apply(lambda x: get_embeddings(x.strip()))

# Convert embeddings into a format suitable for TSNE
matrix = np.array(df['embedding'].tolist())
print(matrix.shape)

tsne = TSNE(n_components=2, perplexity=5, random_state=42, init='random', learning_rate=200)
vis_dims = tsne.fit_transform(matrix)
print(vis_dims)
print(vis_dims.shape)

# Plotting
plt.figure(figsize=(10, 10))
x = vis_dims[:, 0]
y = vis_dims[:, 1]
plt.scatter(x, y, alpha=0.5)

plt.title("Text Embeddings Visualized with t-SNE")
plt.xlabel('Dimension 1')
plt.ylabel('Dimension 2')
plt.show()

content.txt

Text splitting in LangChain is a critical feature that facilitates the division of large texts into smaller, manageable segments. 
This capability is vital for improving comprehension and processing efficiency, especially in tasks that require detailed analysis or extraction of specific contexts.

ChatGPT, developed by OpenAI, represents a leap forward in natural language processing technologies.
It's a conversational AI model capable of understanding and generating human-like text, allowing for dynamic interactions and providing responses that are remarkably coherent and contextually relevant. ChatGPT has been integrated into a multitude of applications, revolutionizing the way we interact with machines and access information.

By leveraging LangChain for text splitting, users can efficiently navigate and analyze vast amounts of text data, facilitating a deeper understanding and more insightful conclusions.
Categories
Tools

Anthropic Tools

from tool_use_package.tools.base_tool import BaseTool
from tool_use_package.tool_user import ToolUser

# 1. Define the Tool
class AdditionTool(BaseTool):
    """Adds together two numbers, a + b."""

    def use_tool(self, a, b):
        print(f"Adding {a} and {b}")
        return a+b

# 2. Tool Description
addition_tool_name = "perform_addition"
addition_tool_description = """Add one number (a) to another (b), returning a+b.
Use this tool WHENEVER you need to perform any addition calculation, as it will ensure your answer is precise."""
addition_tool_parameters = [
    {"name": "a", "type": "float", "description": "The first number in your addition equation."},
    {"name": "b", "type": "float", "description": "The second number in your addition equation."}
]

addition_tool = AdditionTool(addition_tool_name, addition_tool_description, addition_tool_parameters)

# 3. Assign Tool and Ask Claude
tool_user = ToolUser([addition_tool])
messages = [
    {
        "role":"user", 
        "content":"""
            John has three apples. 
            Maggie has nine apples. 
            Tim has 4 bananas. 
            If John gives Maggie 1 apple and 
            Tim gives John 2 bananas, 
            how much total fruits does each person have?"""
    }
]
print(tool_user.use_tools(messages, execution_mode="automatic"))
Categories
Tools

Claude 3 Function Calling

pip install anthropic yfinance rich
export ANTHROPIC_API_KEY=xxxxxxxxxxxx
from anthropic import Anthropic
from rich import print
import re
import yfinance as yf

client = Anthropic()
MODEL_NAME = "claude-3-opus-20240229"

stock_message = {
    "role": "user", 
    "content": "Find the current price of Apple stock"
}

message = client.messages.create(
    model=MODEL_NAME,
    max_tokens=1024,
    messages=[stock_message]
).content[0].text
print("##### Before Function Calling ####\n\n" + message)

# 1. Define the stock price finding function
def get_stock_price(ticker_symbol):
    stock = yf.Ticker(ticker_symbol)
    hist = stock.history(period="1d")
    current_price = hist['Close'].iloc[0]
    return current_price

# 2. Construct Tool description
tool_description = """
<tool_description>
    <tool_name>get_stock_price</tool_name>
    <description>
        Function for finding the current price of a stock using its ticker symbol.
    </description>
    <parameters>
        <parameter>
            <name>ticker_symbol</name>
            <type>str</type>
            <description>Ticker symbol of the stock</description>
        </parameter>
    </parameters>
</tool_description>
"""

# 3. Ask Claude
system_prompt = f"""
In this environment you have access to a set of tools you can use to answer the 
user's question.

You may call them like this:
<function_calls>
    <invoke>
        <tool_name>$TOOL_NAME</tool_name>
        <parameters>
            <$PARAMETER_NAME>$PARAMETER_VALUE</$PARAMETER_NAME>
            ...
        </parameters>
    </invoke>
</function_calls>

Here are the tools available:
<tools>{tool_description}</tools>
"""

function_calling_message = client.messages.create(
    model=MODEL_NAME,
    max_tokens=1024,
    messages=[stock_message],
    system=system_prompt
).content[0].text

# print(function_calling_message)

# 4. Extract parameters from response & call the function
def extract_between_tags(tag, string, strip=False):
    ext_list = re.findall(f"<{tag}>(.+?)</{tag}>", string, re.DOTALL)
    return [e.strip() for e in ext_list] if strip else ext_list

function_params = {"ticker_symbol": extract_between_tags("ticker_symbol", function_calling_message)[0]}
function_name = extract_between_tags("tool_name", function_calling_message)[0]
names_to_functions = {
    'get_stock_price': get_stock_price,
}
price = names_to_functions[function_name](**function_params)

# Construct function results
function_results = f"""
<function_results>
  <result>
    <tool_name>get_stock_price</tool_name>
    <stdout>{price}</stdout>
  </result>
</function_results>"""

# 5. Send all messages back to Claude
partial_assistant_message = function_calling_message + function_results

final_message = client.messages.create(
    model=MODEL_NAME,
    max_tokens=1024,
    messages=[
        stock_message,
        {
            "role": "assistant",
            "content": partial_assistant_message
        }
    ],
    system=system_prompt
).content[0].text
print("\n\n##### After Function Calling #####"+ final_message)

Output

❯ python app.py
##### Before Function Calling ####

To find the current price of Apple stock, I'll use the stock ticker symbol "AAPL" and check a 
financial website or stock market data provider. As of my knowledge cutoff date of September 
2021, the stock price will likely have changed since then. For the most up-to-date price, I 
recommend checking a financial website or app directly.


##### After Function Calling #####

The current price of Apple (AAPL) stock is $175.10.
Categories
AI Agents

CrewAI Meeting Prep

❯ python main.py
## Welcome to the Meeting Prep Crew
-------------------------------
What are the emails for the participants (other than you) in the meeting?
jeff@amazon.com
What is the context of the meeting?
About AI
What is your objective for this meeting?
Launching a new AI Product
Categories
AI Agents

Crew AI Job Posting

git clone https://github.com/joaomdmoura/crewAI-examples
cd crewAI-examples/job-posting
pip install -U 'crewai[tools]'
pip install python-dotenv
export OPENAI_API_KEY=xxxxxxxxxxxxx
export SERPER_API_KEY=xxxxxxxxxxxxx
What is the company description?
Praison AI is a platform specialising in the use of artificial intelligence to automate tasks. Their services likely help users streamline their workflow, generate creative ideas, and potentially automate various processes.
What is the company domain?
https://mer.vin
What are the hiring needs?
AI Engineer
What are specific_benefits you offer?
Remote Work
> Finished chain.
Job Posting Creation Process Completed.
Final Job Posting:
## AI Engineer - Join Our Innovative Team at Praison AI

Praison AI, a pioneering force in AI automation, invites an adept and seasoned AI Engineer to join our dynamic team. We are committed to harnessing the power of artificial intelligence to revolutionize operations and create innovative solutions. If you are ignited by the prospects of AI and seeking a challenging role within a vibrant and progressive environment, your search ends here.

### Role Description:
As an AI Engineer at Praison AI, you will have the privilege to design and implement state-of-the-art AI models, conduct machine learning experiments, and deploy AI-driven applications that simplify tasks. You will be working shoulder-to-shoulder with our data scientists, data engineers, and other stakeholders to infuse AI solutions into our products and services.

### Responsibilities:
- Develop, validate, and deploy cutting-edge AI models.
- Collaborate with the team to design and refine AI prototypes.
- Conduct machine learning experiments and tests, optimizing for efficacy and efficiency.
- Implement the most suitable AI algorithms to meet business needs.
- Oversee the maintenance and management of AI systems and infrastructure.

### Required Skills and Qualifications:
- Bachelor's degree in Computer Science, Engineering, or a related field.
- Proven track record as an AI Engineer or similar role.
- Hands-on experience with machine learning, deep learning, and neural networks.
- Proficiency in AI-focused programming languages such as Python or Java.
- Deep understanding of AI frameworks like TensorFlow or PyTorch.
- Exceptional problem-solving abilities and analytical skills.

### Company Culture and Values:
At Praison AI, we champion innovation, collaboration, and the relentless pursuit of excellence. We have cultivated an environment that motivates continuous learning and growth. We believe that diversity of thought is the recipe for success and are committed to building an inclusive and engaging workplace for all our employees.

### Unique Benefits:
Praison AI is more than just a workplace—it's a community of like-minded individuals pushing the boundaries of what's possible with AI. Along with competitive salaries and comprehensive healthcare coverage, we offer the flexibility of work arrangements. We are dedicated to providing a continuous learning environment that encourages professional growth and development.

As we strive for innovation and excellence, we offer opportunities to work on exciting, cutting-edge projects in the AI field. Plus, the flexibility to work from anywhere as we believe in promoting a healthy work-life balance.

To apply, email your resume, cover letter, and any relevant portfolio links to careers@praisoai.com with the subject 'AI Engineer Application'. We can't wait to hear from you!

Praison AI is a proud equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees.

job_posting.md

## AI Engineer - Join Our Innovative Team at Praison AI

Praison AI, a pioneering force in AI automation, invites an adept and seasoned AI Engineer to join our dynamic team. We are committed to harnessing the power of artificial intelligence to revolutionize operations and create innovative solutions. If you are ignited by the prospects of AI and seeking a challenging role within a vibrant and progressive environment, your search ends here.

### Role Description:
As an AI Engineer at Praison AI, you will have the privilege to design and implement state-of-the-art AI models, conduct machine learning experiments, and deploy AI-driven applications that simplify tasks. You will be working shoulder-to-shoulder with our data scientists, data engineers, and other stakeholders to infuse AI solutions into our products and services.

### Responsibilities:
- Develop, validate, and deploy cutting-edge AI models.
- Collaborate with the team to design and refine AI prototypes.
- Conduct machine learning experiments and tests, optimizing for efficacy and efficiency.
- Implement the most suitable AI algorithms to meet business needs.
- Oversee the maintenance and management of AI systems and infrastructure.

### Required Skills and Qualifications:
- Bachelor's degree in Computer Science, Engineering, or a related field.
- Proven track record as an AI Engineer or similar role.
- Hands-on experience with machine learning, deep learning, and neural networks.
- Proficiency in AI-focused programming languages such as Python or Java.
- Deep understanding of AI frameworks like TensorFlow or PyTorch.
- Exceptional problem-solving abilities and analytical skills.

### Company Culture and Values:
At Praison AI, we champion innovation, collaboration, and the relentless pursuit of excellence. We have cultivated an environment that motivates continuous learning and growth. We believe that diversity of thought is the recipe for success and are committed to building an inclusive and engaging workplace for all our employees.

### Unique Benefits:
Praison AI is more than just a workplace—it's a community of like-minded individuals pushing the boundaries of what's possible with AI. Along with competitive salaries and comprehensive healthcare coverage, we offer the flexibility of work arrangements. We are dedicated to providing a continuous learning environment that encourages professional growth and development.

As we strive for innovation and excellence, we offer opportunities to work on exciting, cutting-edge projects in the AI field. Plus, the flexibility to work from anywhere as we believe in promoting a healthy work-life balance.

To apply, email your resume, cover letter, and any relevant portfolio links to careers@praisoai.com with the subject 'AI Engineer Application'. We can't wait to hear from you!

Praison AI is a proud equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees.
export OPENAI_API_BASE=https://api.groq.com/openai/v1
export OPENAI_API_KEY=gsk_xxxxxxxx
export OPENAI_MODEL_NAME=mixtral-8x7b-32768
export OPENAI_MODEL_NAME=llama2-70b-4096
Categories
Embedding

Advanced Chunking Strategies

pip install -U chromadb langchain llama-index langchain_experimental langchain_openai
from rich import print
from langchain.docstore.document import Document
from langchain_community.chat_models import ChatOllama
from langchain_community.vectorstores import Chroma
from langchain_community import embeddings
from langchain_core.runnables import RunnablePassthrough
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate

local_llm = ChatOllama(model="mistral")

# RAG
def rag(chunks, collection_name):
    vectorstore = Chroma.from_documents(
        documents=documents,
        collection_name=collection_name,
        embedding=embeddings.ollama.OllamaEmbeddings(model='nomic-embed-text'),
    )
    retriever = vectorstore.as_retriever()

    prompt_template = """Answer the question based only on the following context:
    {context}
    Question: {question}
    """
    prompt = ChatPromptTemplate.from_template(prompt_template)

    chain = (
        {"context": retriever, "question": RunnablePassthrough()}
        | prompt
        | local_llm
        | StrOutputParser()
    )
    result = chain.invoke("What is the use of Text Splitting?")
    print(result)


# 1. Character Text Splitting
print("#### Character Text Splitting ####")

text = "Text splitting in LangChain is a critical feature that facilitates the division of large texts into smaller, manageable segments. "

# Manual Splitting
chunks = []
chunk_size = 35 # Characters
for i in range(0, len(text), chunk_size):
    chunk = text[i:i + chunk_size]
    chunks.append(chunk)
documents = [Document(page_content=chunk, metadata={"source": "local"}) for chunk in chunks]
print(documents)

# Automatic Text Splitting
from langchain.text_splitter import CharacterTextSplitter
text_splitter = CharacterTextSplitter(chunk_size = 35, chunk_overlap=0, separator='', strip_whitespace=False)
documents = text_splitter.create_documents([text])
print(documents)

# 2. Recursive Character Text Splitting
print("#### Recursive Character Text Splitting ####")

from langchain.text_splitter import RecursiveCharacterTextSplitter
with open('content.txt', 'r', encoding='utf-8') as file:
    text = file.read()

text_splitter = RecursiveCharacterTextSplitter(chunk_size = 65, chunk_overlap=0) # ["\n\n", "\n", " ", ""] 65,450
print(text_splitter.create_documents([text])) 

# 3. Document Specific Splitting
print("#### Document Specific Splitting ####")

# Document Specific Splitting - Markdown
from langchain.text_splitter import MarkdownTextSplitter
splitter = MarkdownTextSplitter(chunk_size = 40, chunk_overlap=0)
markdown_text = """
# Fun in California

## Driving

Try driving on the 1 down to San Diego

### Food

Make sure to eat a burrito while you're there

## Hiking

Go to Yosemite
"""
print(splitter.create_documents([markdown_text]))

# Document Specific Splitting - Python
from langchain.text_splitter import PythonCodeTextSplitter
python_text = """
class Person:
  def __init__(self, name, age):
    self.name = name
    self.age = age

p1 = Person("John", 36)

for i in range(10):
    print (i)
"""
python_splitter = PythonCodeTextSplitter(chunk_size=100, chunk_overlap=0)
print(python_splitter.create_documents([python_text]))

# Document Specific Splitting - Javascript
from langchain.text_splitter import RecursiveCharacterTextSplitter, Language
javascript_text = """
// Function is called, the return value will end up in x
let x = myFunction(4, 3);

function myFunction(a, b) {
// Function returns the product of a and b
  return a * b;
}
"""
js_splitter = RecursiveCharacterTextSplitter.from_language(
    language=Language.JS, chunk_size=65, chunk_overlap=0
)
print(js_splitter.create_documents([javascript_text]))

# 4. Semantic Chunking
print("#### Semantic Chunking ####")

from langchain_experimental.text_splitter import SemanticChunker
from langchain_openai.embeddings import OpenAIEmbeddings

# Percentile - all differences between sentences are calculated, and then any difference greater than the X percentile is split
text_splitter = SemanticChunker(OpenAIEmbeddings())
text_splitter = SemanticChunker(
    OpenAIEmbeddings(), breakpoint_threshold_type="percentile" # "standard_deviation", "interquartile"
)
documents = text_splitter.create_documents([text])
print(documents)

# 5. Agentic Chunking
print("#### Proposition-Based Chunking ####")

# https://arxiv.org/pdf/2312.06648.pdf

from langchain.output_parsers.openai_tools import JsonOutputToolsParser
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnableLambda
from langchain.chains import create_extraction_chain
from typing import Optional, List
from langchain.chains import create_extraction_chain_pydantic
from langchain_core.pydantic_v1 import BaseModel
from langchain import hub

obj = hub.pull("wfh/proposal-indexing")
llm = ChatOpenAI(model='gpt-3.5-turbo')
runnable = obj | llm

class Sentences(BaseModel):
    sentences: List[str]
    
# Extraction
extraction_chain = create_extraction_chain_pydantic(pydantic_schema=Sentences, llm=llm)
def get_propositions(text):
    runnable_output = runnable.invoke({
    	"input": text
    }).content
    propositions = extraction_chain.invoke(runnable_output)["text"][0].sentences
    return propositions
    
paragraphs = text.split("\n\n")
text_propositions = []
for i, para in enumerate(paragraphs[:5]):
    propositions = get_propositions(para)
    text_propositions.extend(propositions)
    print (f"Done with {i}")

print (f"You have {len(text_propositions)} propositions")
print(text_propositions[:10])

print("#### Agentic Chunking ####")

from agentic_chunker import AgenticChunker
ac = AgenticChunker()
ac.add_propositions(text_propositions)
print(ac.pretty_print_chunks())
chunks = ac.get_chunks(get_type='list_of_strings')
print(chunks)
documents = [Document(page_content=chunk, metadata={"source": "local"}) for chunk in chunks]
rag(documents, "agentic-chunks")

agentic_chunker.py

from langchain_core.prompts import ChatPromptTemplate
import uuid
from langchain_openai import ChatOpenAI
import os
from typing import Optional
from langchain_core.pydantic_v1 import BaseModel
from langchain.chains import create_extraction_chain_pydantic
from dotenv import load_dotenv
from rich import print

load_dotenv()

class AgenticChunker:
    def __init__(self, openai_api_key=None):
        self.chunks = {}
        self.id_truncate_limit = 5

        # Whether or not to update/refine summaries and titles as you get new information
        self.generate_new_metadata_ind = True
        self.print_logging = True

        if openai_api_key is None:
            openai_api_key = os.getenv("OPENAI_API_KEY")

        if openai_api_key is None:
            raise ValueError("API key is not provided and not found in environment variables")

        self.llm = ChatOpenAI(model='gpt-3.5-turbo', openai_api_key=openai_api_key, temperature=0)

    def add_propositions(self, propositions):
        for proposition in propositions:
            self.add_proposition(proposition)
    
    def add_proposition(self, proposition):
        if self.print_logging:
            print (f"\nAdding: '{proposition}'")

        # If it's your first chunk, just make a new chunk and don't check for others
        if len(self.chunks) == 0:
            if self.print_logging:
                print ("No chunks, creating a new one")
            self._create_new_chunk(proposition)
            return

        chunk_id = self._find_relevant_chunk(proposition)

        # If a chunk was found then add the proposition to it
        if chunk_id:
            if self.print_logging:
                print (f"Chunk Found ({self.chunks[chunk_id]['chunk_id']}), adding to: {self.chunks[chunk_id]['title']}")
            self.add_proposition_to_chunk(chunk_id, proposition)
            return
        else:
            if self.print_logging:
                print ("No chunks found")
            # If a chunk wasn't found, then create a new one
            self._create_new_chunk(proposition)
        

    def add_proposition_to_chunk(self, chunk_id, proposition):
        # Add then
        self.chunks[chunk_id]['propositions'].append(proposition)

        # Then grab a new summary
        if self.generate_new_metadata_ind:
            self.chunks[chunk_id]['summary'] = self._update_chunk_summary(self.chunks[chunk_id])
            self.chunks[chunk_id]['title'] = self._update_chunk_title(self.chunks[chunk_id])

    def _update_chunk_summary(self, chunk):
        """
        If you add a new proposition to a chunk, you may want to update the summary or else they could get stale
        """
        PROMPT = ChatPromptTemplate.from_messages(
            [
                (
                    "system",
                    """
                    You are the steward of a group of chunks which represent groups of sentences that talk about a similar topic
                    A new proposition was just added to one of your chunks, you should generate a very brief 1-sentence summary which will inform viewers what a chunk group is about.

                    A good summary will say what the chunk is about, and give any clarifying instructions on what to add to the chunk.

                    You will be given a group of propositions which are in the chunk and the chunks current summary.

                    Your summaries should anticipate generalization. If you get a proposition about apples, generalize it to food.
                    Or month, generalize it to "date and times".

                    Example:
                    Input: Proposition: Greg likes to eat pizza
                    Output: This chunk contains information about the types of food Greg likes to eat.

                    Only respond with the chunk new summary, nothing else.
                    """,
                ),
                ("user", "Chunk's propositions:\n{proposition}\n\nCurrent chunk summary:\n{current_summary}"),
            ]
        )

        runnable = PROMPT | self.llm

        new_chunk_summary = runnable.invoke({
            "proposition": "\n".join(chunk['propositions']),
            "current_summary" : chunk['summary']
        }).content

        return new_chunk_summary
    
    def _update_chunk_title(self, chunk):
        """
        If you add a new proposition to a chunk, you may want to update the title or else it can get stale
        """
        PROMPT = ChatPromptTemplate.from_messages(
            [
                (
                    "system",
                    """
                    You are the steward of a group of chunks which represent groups of sentences that talk about a similar topic
                    A new proposition was just added to one of your chunks, you should generate a very brief updated chunk title which will inform viewers what a chunk group is about.

                    A good title will say what the chunk is about.

                    You will be given a group of propositions which are in the chunk, chunk summary and the chunk title.

                    Your title should anticipate generalization. If you get a proposition about apples, generalize it to food.
                    Or month, generalize it to "date and times".

                    Example:
                    Input: Summary: This chunk is about dates and times that the author talks about
                    Output: Date & Times

                    Only respond with the new chunk title, nothing else.
                    """,
                ),
                ("user", "Chunk's propositions:\n{proposition}\n\nChunk summary:\n{current_summary}\n\nCurrent chunk title:\n{current_title}"),
            ]
        )

        runnable = PROMPT | self.llm

        updated_chunk_title = runnable.invoke({
            "proposition": "\n".join(chunk['propositions']),
            "current_summary" : chunk['summary'],
            "current_title" : chunk['title']
        }).content

        return updated_chunk_title

    def _get_new_chunk_summary(self, proposition):
        PROMPT = ChatPromptTemplate.from_messages(
            [
                (
                    "system",
                    """
                    You are the steward of a group of chunks which represent groups of sentences that talk about a similar topic
                    You should generate a very brief 1-sentence summary which will inform viewers what a chunk group is about.

                    A good summary will say what the chunk is about, and give any clarifying instructions on what to add to the chunk.

                    You will be given a proposition which will go into a new chunk. This new chunk needs a summary.

                    Your summaries should anticipate generalization. If you get a proposition about apples, generalize it to food.
                    Or month, generalize it to "date and times".

                    Example:
                    Input: Proposition: Greg likes to eat pizza
                    Output: This chunk contains information about the types of food Greg likes to eat.

                    Only respond with the new chunk summary, nothing else.
                    """,
                ),
                ("user", "Determine the summary of the new chunk that this proposition will go into:\n{proposition}"),
            ]
        )

        runnable = PROMPT | self.llm

        new_chunk_summary = runnable.invoke({
            "proposition": proposition
        }).content

        return new_chunk_summary
    
    def _get_new_chunk_title(self, summary):
        PROMPT = ChatPromptTemplate.from_messages(
            [
                (
                    "system",
                    """
                    You are the steward of a group of chunks which represent groups of sentences that talk about a similar topic
                    You should generate a very brief few word chunk title which will inform viewers what a chunk group is about.

                    A good chunk title is brief but encompasses what the chunk is about

                    You will be given a summary of a chunk which needs a title

                    Your titles should anticipate generalization. If you get a proposition about apples, generalize it to food.
                    Or month, generalize it to "date and times".

                    Example:
                    Input: Summary: This chunk is about dates and times that the author talks about
                    Output: Date & Times

                    Only respond with the new chunk title, nothing else.
                    """,
                ),
                ("user", "Determine the title of the chunk that this summary belongs to:\n{summary}"),
            ]
        )

        runnable = PROMPT | self.llm

        new_chunk_title = runnable.invoke({
            "summary": summary
        }).content

        return new_chunk_title


    def _create_new_chunk(self, proposition):
        new_chunk_id = str(uuid.uuid4())[:self.id_truncate_limit] # I don't want long ids
        new_chunk_summary = self._get_new_chunk_summary(proposition)
        new_chunk_title = self._get_new_chunk_title(new_chunk_summary)

        self.chunks[new_chunk_id] = {
            'chunk_id' : new_chunk_id,
            'propositions': [proposition],
            'title' : new_chunk_title,
            'summary': new_chunk_summary,
            'chunk_index' : len(self.chunks)
        }
        if self.print_logging:
            print (f"Created new chunk ({new_chunk_id}): {new_chunk_title}")
    
    def get_chunk_outline(self):
        """
        Get a string which represents the chunks you currently have.
        This will be empty when you first start off
        """
        chunk_outline = ""

        for chunk_id, chunk in self.chunks.items():
            single_chunk_string = f"""Chunk ({chunk['chunk_id']}): {chunk['title']}\nSummary: {chunk['summary']}\n\n"""
        
            chunk_outline += single_chunk_string
        
        return chunk_outline

    def _find_relevant_chunk(self, proposition):
        current_chunk_outline = self.get_chunk_outline()

        PROMPT = ChatPromptTemplate.from_messages(
            [
                (
                    "system",
                    """
                    Determine whether or not the "Proposition" should belong to any of the existing chunks.

                    A proposition should belong to a chunk of their meaning, direction, or intention are similar.
                    The goal is to group similar propositions and chunks.

                    If you think a proposition should be joined with a chunk, return the chunk id.
                    If you do not think an item should be joined with an existing chunk, just return "No chunks"

                    Example:
                    Input:
                        - Proposition: "Greg really likes hamburgers"
                        - Current Chunks:
                            - Chunk ID: 2n4l3d
                            - Chunk Name: Places in San Francisco
                            - Chunk Summary: Overview of the things to do with San Francisco Places

                            - Chunk ID: 93833k
                            - Chunk Name: Food Greg likes
                            - Chunk Summary: Lists of the food and dishes that Greg likes
                    Output: 93833k
                    """,
                ),
                ("user", "Current Chunks:\n--Start of current chunks--\n{current_chunk_outline}\n--End of current chunks--"),
                ("user", "Determine if the following statement should belong to one of the chunks outlined:\n{proposition}"),
            ]
        )

        runnable = PROMPT | self.llm

        chunk_found = runnable.invoke({
            "proposition": proposition,
            "current_chunk_outline": current_chunk_outline
        }).content

        # Pydantic data class
        class ChunkID(BaseModel):
            """Extracting the chunk id"""
            chunk_id: Optional[str]
            
        # Extraction to catch-all LLM responses. This is a bandaid
        extraction_chain = create_extraction_chain_pydantic(pydantic_schema=ChunkID, llm=self.llm)
        extraction_found = extraction_chain.invoke(chunk_found)["text"]
        if extraction_found:
            chunk_found = extraction_found[0].chunk_id

        # If you got a response that isn't the chunk id limit, chances are it's a bad response or it found nothing
        # So return nothing
        if len(chunk_found) != self.id_truncate_limit:
            return None

        return chunk_found
    
    def get_chunks(self, get_type='dict'):
        """
        This function returns the chunks in the format specified by the 'get_type' parameter.
        If 'get_type' is 'dict', it returns the chunks as a dictionary.
        If 'get_type' is 'list_of_strings', it returns the chunks as a list of strings, where each string is a proposition in the chunk.
        """
        if get_type == 'dict':
            return self.chunks
        if get_type == 'list_of_strings':
            chunks = []
            for chunk_id, chunk in self.chunks.items():
                chunks.append(" ".join([x for x in chunk['propositions']]))
            return chunks
    
    def pretty_print_chunks(self):
        print (f"\nYou have {len(self.chunks)} chunks\n")
        for chunk_id, chunk in self.chunks.items():
            print(f"Chunk #{chunk['chunk_index']}")
            print(f"Chunk ID: {chunk_id}")
            print(f"Summary: {chunk['summary']}")
            print(f"Propositions:")
            for prop in chunk['propositions']:
                print(f"    -{prop}")
            print("\n\n")

    def pretty_print_chunk_outline(self):
        print ("Chunk Outline\n")
        print(self.get_chunk_outline())

if __name__ == "__main__":
    ac = AgenticChunker()

    ## Comment and uncomment the propositions to your hearts content
    propositions = [
        'The month is October.',
        'The year is 2023.',
        "One of the most important things that I didn't understand about the world as a child was the degree to which the returns for performance are superlinear.",
        'Teachers and coaches implicitly told us that the returns were linear.',
        "I heard a thousand times that 'You get out what you put in.'",
        # 'Teachers and coaches meant well.',
        # "The statement that 'You get out what you put in' is rarely true.",
        # "If your product is only half as good as your competitor's product, you do not get half as many customers.",
        # "You get no customers if your product is only half as good as your competitor's product.",
        # 'You go out of business if you get no customers.',
        # 'The returns for performance are superlinear in business.',
        # 'Some people think the superlinear returns for performance are a flaw of capitalism.',
        # 'Some people think that changing the rules of capitalism would stop the superlinear returns for performance from being true.',
        # 'Superlinear returns for performance are a feature of the world.',
        # 'Superlinear returns for performance are not an artifact of rules that humans have invented.',
        # 'The same pattern of superlinear returns is observed in fame.',
        # 'The same pattern of superlinear returns is observed in power.',
        # 'The same pattern of superlinear returns is observed in military victories.',
        # 'The same pattern of superlinear returns is observed in knowledge.',
        # 'The same pattern of superlinear returns is observed in benefit to humanity.',
        # 'In fame, power, military victories, knowledge, and benefit to humanity, the rich get richer.'
    ]
    
    ac.add_propositions(propositions)
    ac.pretty_print_chunks()
    ac.pretty_print_chunk_outline()
    print (ac.get_chunks(get_type='list_of_strings'))

content.txt

Text splitting in LangChain is a critical feature that facilitates the division of large texts into smaller, manageable segments. 
This capability is vital for improving comprehension and processing efficiency, especially in tasks that require detailed analysis or extraction of specific contexts.

ChatGPT, developed by OpenAI, represents a leap forward in natural language processing technologies.
It's a conversational AI model capable of understanding and generating human-like text, allowing for dynamic interactions and providing responses that are remarkably coherent and contextually relevant. ChatGPT has been integrated into a multitude of applications, revolutionizing the way we interact with machines and access information.

By leveraging LangChain for text splitting, users can efficiently navigate and analyze vast amounts of text data, facilitating a deeper understanding and more insightful conclusions.

Output

❯ python app.py
#### Character Text Splitting ####
[
    Document(page_content='Text splitting in LangChain is a cr', metadata={'source': 'local'}),
    Document(page_content='itical feature that facilitates the', metadata={'source': 'local'}),
    Document(page_content=' division of large texts into small', metadata={'source': 'local'}),
    Document(page_content='er, manageable segments. ', metadata={'source': 'local'})
]
[
    Document(page_content='Text splitting in LangChain is a cr'),
    Document(page_content='itical feature that facilitates the'),
    Document(page_content=' division of large texts into small'),
    Document(page_content='er, manageable segments. ')
]
#### Recursive Character Text Splitting ####
[
    Document(page_content='Text splitting in LangChain is a critical feature that'),
    Document(page_content='facilitates the division of large texts into smaller, manageable'),
    Document(page_content='segments.'),
    Document(page_content='This capability is vital for improving comprehension and'),
    Document(page_content='processing efficiency, especially in tasks that require detailed'),
    Document(page_content='analysis or extraction of specific contexts.'),
    Document(page_content='ChatGPT, developed by OpenAI, represents a leap forward in'),
    Document(page_content='natural language processing technologies.'),
    Document(page_content="It's a conversational AI model capable of understanding and"),
    Document(page_content='generating human-like text, allowing for dynamic interactions'),
    Document(page_content='and providing responses that are remarkably coherent and'),
    Document(page_content='contextually relevant. ChatGPT has been integrated into a'),
    Document(page_content='multitude of applications, revolutionizing the way we interact'),
    Document(page_content='with machines and access information.'),
    Document(page_content='By leveraging LangChain for text splitting, users can'),
    Document(page_content='efficiently navigate and analyze vast amounts of text data,'),
    Document(page_content='facilitating a deeper understanding and more insightful'),
    Document(page_content='conclusions.')
]
#### Document Specific Splitting ####
[
    Document(page_content='# Fun in California\n\n## Driving'),
    Document(page_content='Try driving on the 1 down to San Diego'),
    Document(page_content='### Food'),
    Document(page_content="Make sure to eat a burrito while you're"),
    Document(page_content='there'),
    Document(page_content='## Hiking\n\nGo to Yosemite')
]
[
    Document(page_content='class Person:\n  def __init__(self, name, age):\n    self.name = name\n    self.age = age'),
    Document(page_content='p1 = Person("John", 36)\n\nfor i in range(10):\n    print (i)')
]
[
    Document(page_content='// Function is called, the return value will end up in x'),
    Document(page_content='let x = myFunction(4, 3);'),
    Document(page_content='function myFunction(a, b) {'),
    Document(page_content='// Function returns the product of a and b\n  return a * b;\n}')
]
#### Semantic Chunking ####
[
    Document(
        page_content='Text splitting in LangChain is a critical feature that facilitates the division of large texts into 
smaller, manageable segments. This capability is vital for improving comprehension and processing efficiency, especially in tasks
that require detailed analysis or extraction of specific contexts.'
    ),
    Document(
        page_content="ChatGPT, developed by OpenAI, represents a leap forward in natural language processing technologies. It's a
conversational AI model capable of understanding and generating human-like text, allowing for dynamic interactions and providing 
responses that are remarkably coherent and contextually relevant. ChatGPT has been integrated into a multitude of applications, 
revolutionizing the way we interact with machines and access information. By leveraging LangChain for text splitting, users can 
efficiently navigate and analyze vast amounts of text data, facilitating a deeper understanding and more insightful conclusions."
    )
]
#### Proposition-Based Chunking ####
Done with 0
Done with 1
Done with 2
You have 17 propositions
[
    'Text splitting in LangChain is a critical feature.',
    'Text splitting facilitates the division of large texts into smaller, manageable segments.',
    'This capability is vital for improving comprehension and processing efficiency.',
    'It is especially important in tasks that require detailed analysis or extraction of specific contexts.',
    'ChatGPT was developed by OpenAI.',
    'OpenAI developed ChatGPT.',
    'ChatGPT represents a leap forward in natural language processing technologies.',
    'ChatGPT is a conversational AI model.',
    'ChatGPT is capable of understanding and generating human-like text.',
    'ChatGPT allows for dynamic interactions.'
]
#### Agentic Chunking ####

Adding: 'Text splitting in LangChain is a critical feature.'
No chunks, creating a new one
Created new chunk (0e05f): LangChain Features

Adding: 'Text splitting facilitates the division of large texts into smaller, manageable segments.'
No chunks found
Created new chunk (471d6): Text Segmentation Techniques

Adding: 'This capability is vital for improving comprehension and processing efficiency.'
No chunks found
Created new chunk (9ba91): Capabilities Importance & Benefits

Adding: 'It is especially important in tasks that require detailed analysis or extraction of specific contexts.'
No chunks found
Created new chunk (3af8b): Analytical Processes

Adding: 'ChatGPT was developed by OpenAI.'
No chunks found
Created new chunk (e2947): ChatGPT Development & Features

Adding: 'OpenAI developed ChatGPT.'
Chunk Found (e2947), adding to: ChatGPT Development & Features

Adding: 'ChatGPT represents a leap forward in natural language processing technologies.'
Chunk Found (e2947), adding to: ChatGPT Development

Adding: 'ChatGPT is a conversational AI model.'
Chunk Found (e2947), adding to: Advancements in Natural Language Processing

Adding: 'ChatGPT is capable of understanding and generating human-like text.'
Chunk Found (e2947), adding to: ChatGPT: Development and Capabilities

Adding: 'ChatGPT allows for dynamic interactions.'
Chunk Found (e2947), adding to: ChatGPT: Development, Capabilities, and Significance

Adding: 'ChatGPT provides responses that are remarkably coherent and contextually relevant.'
Chunk Found (e2947), adding to: ChatGPT: Overview and Innovations

Adding: 'ChatGPT has been integrated into a multitude of applications.'
Chunk Found (e2947), adding to: ChatGPT: Development, Capabilities & Impact

Adding: 'ChatGPT revolutionized the way we interact with machines.'
Chunk Found (e2947), adding to: ChatGPT: Overview & Applications

Adding: 'ChatGPT revolutionized the way we access information.'
Chunk Found (e2947), adding to: ChatGPT: Development, Capabilities & Impact

Adding: 'Users can leverage LangChain for text splitting.'
Chunk Found (0e05f), adding to: LangChain Features

Adding: 'LangChain allows users to efficiently navigate and analyze vast amounts of text data.'
Chunk Found (0e05f), adding to: Using LangChain for Text Splitting

Adding: 'Text splitting with LangChain facilitates a deeper understanding and more insightful conclusions.'
Chunk Found (0e05f), adding to: LangChain Text Splitting and Analysis

You have 5 chunks

Chunk #0
Chunk ID: 0e05f
Summary: This chunk contains information about using LangChain for text splitting, including its advantages for navigating, 
analyzing, and understanding large text datasets.
Propositions:
    -Text splitting in LangChain is a critical feature.
    -Users can leverage LangChain for text splitting.
    -LangChain allows users to efficiently navigate and analyze vast amounts of text data.
    -Text splitting with LangChain facilitates a deeper understanding and more insightful conclusions.



Chunk #1
Chunk ID: 471d6
Summary: This chunk contains information about techniques and methods for dividing texts into smaller segments.
Propositions:
    -Text splitting facilitates the division of large texts into smaller, manageable segments.



Chunk #2
Chunk ID: 9ba91
Summary: This chunk contains information about the importance and benefits of certain capabilities.
Propositions:
    -This capability is vital for improving comprehension and processing efficiency.



Chunk #3
Chunk ID: 3af8b
Summary: This chunk contains information about the importance of certain processes in tasks requiring detailed analysis or 
context extraction.
Propositions:
    -It is especially important in tasks that require detailed analysis or extraction of specific contexts.



Chunk #4
Chunk ID: e2947
Summary: This chunk contains information about the development, capabilities, significance, functionalities, and applications of 
ChatGPT, a conversational AI model by OpenAI.
Propositions:
    -ChatGPT was developed by OpenAI.
    -OpenAI developed ChatGPT.
    -ChatGPT represents a leap forward in natural language processing technologies.
    -ChatGPT is a conversational AI model.
    -ChatGPT is capable of understanding and generating human-like text.
    -ChatGPT allows for dynamic interactions.
    -ChatGPT provides responses that are remarkably coherent and contextually relevant.
    -ChatGPT has been integrated into a multitude of applications.
    -ChatGPT revolutionized the way we interact with machines.
    -ChatGPT revolutionized the way we access information.



None
[
    'Text splitting in LangChain is a critical feature. Users can leverage LangChain for text splitting. LangChain allows users 
to efficiently navigate and analyze vast amounts of text data. Text splitting with LangChain facilitates a deeper understanding 
and more insightful conclusions.',
    'Text splitting facilitates the division of large texts into smaller, manageable segments.',
    'This capability is vital for improving comprehension and processing efficiency.',
    'It is especially important in tasks that require detailed analysis or extraction of specific contexts.',
    'ChatGPT was developed by OpenAI. OpenAI developed ChatGPT. ChatGPT represents a leap forward in natural language processing 
technologies. ChatGPT is a conversational AI model. ChatGPT is capable of understanding and generating human-like text. ChatGPT 
allows for dynamic interactions. ChatGPT provides responses that are remarkably coherent and contextually relevant. ChatGPT has 
been integrated into a multitude of applications. ChatGPT revolutionized the way we interact with machines. ChatGPT 
revolutionized the way we access information.'
]
 Text splitting is a feature used to divide large texts into smaller, manageable segments. This facilitates improved 
comprehension and processing efficiency, making it especially important in tasks that require detailed analysis or extraction of 
specific contexts. It enables users to more efficiently navigate and analyze vast amounts of text data, leading to deeper 
understanding and more insightful conclusions.
Categories
AutoGen

AutoGen Graph

pip install -U "pyautogen[graph]>=0.2.11" matplotlib networkx
import random
import matplotlib.pyplot as plt
import networkx as nx
import autogen
from autogen.agentchat.conversable_agent import ConversableAgent
from autogen.agentchat.assistant_agent import AssistantAgent
from autogen.agentchat.groupchat import GroupChat
from autogen.graph_utils import visualize_speaker_transitions_dict

config_list_gpt4 = {
    "timeout": 600,
    "cache_seed": 44,
    "temperature": 0,
    "config_list": [{"model": "gpt-4-turbo-preview"}],
}

def get_agent_of_name(agents, name) -> ConversableAgent:
    for agent in agents:
        if agent.name == name:
            return agent

agents = []
speaker_transitions_dict = {}
secret_values = {}

for prefix in ["A", "B", "C"]:
    for i in range(3):
        node_id = f"{prefix}{i}"
        secret_value = random.randint(1, 5)
        secret_values[node_id] = secret_value
        agents.append(
            AssistantAgent(
                name=node_id,
                system_message=f"""Your name is {node_id}.
Do not respond as the speaker named in the NEXT tag if your name is not in the NEXT tag. Instead, suggest a relevant team leader to handle the mis-tag, with the NEXT: tag.

You have {secret_value} chocolates.

The list of players are [A0, A1, A2, B0, B1, B2, C0, C1, C2].

Your first character of your name is your team, and your second character denotes that you are a team leader if it is 0.
CONSTRAINTS: Team members can only talk within the team, whilst team leader can talk to team leaders of other teams but not team members of other teams.

You can use NEXT: to suggest the next speaker. You have to respect the CONSTRAINTS, and can only suggest one player from the list of players, i.e., do not suggest A3 because A3 is not from the list of players.
Team leaders must make sure that they know the sum of the individual chocolate count of all three players in their own team, i.e., A0 is responsible for team A only.

Keep track of the player's tally using a JSON format so that others can check the total tally. Use
A0:?, A1:?, A2:?,
B0:?, B1:?, B2:?,
C0:?, C1:?, C2:?

If you are the team leader, you should aggregate your team's total chocolate count to cooperate.
Once the team leader know their team's tally, they can suggest another team leader for them to find their team tally, because we need all three team tallys to succeed.
Use NEXT: to suggest the next speaker, e.g., NEXT: A0.

Once we have the total tally from all nine players, sum up all three teams' tally, then terminate the discussion using TERMINATE.
""",
                llm_config=config_list_gpt4,
            )
        )
        speaker_transitions_dict[agents[-1]] = []

for prefix in ["A", "B", "C"]:
    for i in range(3):
        source_id = f"{prefix}{i}"
        for j in range(3):
            target_id = f"{prefix}{j}"
            if i != j:
                speaker_transitions_dict[get_agent_of_name(agents, source_id)].append(get_agent_of_name(agents, target_id))

speaker_transitions_dict[get_agent_of_name(agents, "A0")].append(get_agent_of_name(agents, "B0"))
speaker_transitions_dict[get_agent_of_name(agents, "A0")].append(get_agent_of_name(agents, "C0"))
speaker_transitions_dict[get_agent_of_name(agents, "B0")].append(get_agent_of_name(agents, "A0"))
speaker_transitions_dict[get_agent_of_name(agents, "B0")].append(get_agent_of_name(agents, "C0"))
speaker_transitions_dict[get_agent_of_name(agents, "C0")].append(get_agent_of_name(agents, "A0"))
speaker_transitions_dict[get_agent_of_name(agents, "C0")].append(get_agent_of_name(agents, "B0"))

graph = nx.DiGraph()
graph.add_nodes_from([agent.name for agent in agents])
for key, value in speaker_transitions_dict.items():
    for agent in value:
        graph.add_edge(key.name, agent.name)

plt.figure(figsize=(12, 10))
pos = nx.spring_layout(graph)
nx.draw(graph, pos, with_labels=True, font_weight="bold")
for node, (x, y) in pos.items():
    secret_value = secret_values[node]
    plt.text(x, y + 0.1, s=f"Secret: {secret_value}", horizontalalignment="center")
plt.show()

def is_termination_msg(content) -> bool:
    have_content = content.get("content", None) is not None
    if have_content and "TERMINATE" in content["content"]:
        return True
    return False

user_proxy = autogen.UserProxyAgent(
    name="User_proxy",
    system_message="Terminator admin.",
    code_execution_config=False,
    is_termination_msg=is_termination_msg,
    human_input_mode="NEVER",
)

agents.append(user_proxy)

group_chat = GroupChat(
    agents=agents,
    messages=[],
    max_round=20,
    allowed_or_disallowed_speaker_transitions=speaker_transitions_dict,
    speaker_transitions_type="allowed",
)

manager = autogen.GroupChatManager(
    groupchat=group_chat,
    llm_config=config_list_gpt4,
    code_execution_config=False,
    is_termination_msg=is_termination_msg,
)

agents[0].initiate_chat(
    manager,
    message="""
There are 9 players in this game, split equally into Teams A, B, C. Therefore each team has 3 players, including the team leader.
The task is to find out the sum of chocolate count from all nine players. I will now start with my team.
NEXT: A1""",
)

Output

❯ python graph.py
WARNING:root:Warning: There are isolated agent nodes, there are not incoming nor outgoing edges. Isolated agents: ['User_proxy']
WARNING:root:Warning: The set of agents in allowed_speaker_transitions do not match agents. Offending agents: ['User_proxy']
A0 (to chat_manager):


There are 9 players in this game, split equally into Teams A, B, C. Therefore each team has 3 players, including the team leader.
The task is to find out the sum of chocolate count from all nine players. I will now start with my team.
NEXT: A1

--------------------------------------------------------------------------------
A1 (to chat_manager):

A0:?, A1:4, A2:?,
B0:?, B1:?, B2:?,
C0:?, C1:?, C2:?

I have 4 chocolates. Let's find out how many chocolates A2 has.
NEXT: A2

--------------------------------------------------------------------------------
A2 (to chat_manager):

A0:?, A1:4, A2:5,
B0:?, B1:?, B2:?,
C0:?, C1:?, C2:?

I have 5 chocolates. Now, we need to report back to our team leader, A0, to aggregate our team's total chocolate count.
NEXT: A0

--------------------------------------------------------------------------------
A0 (to chat_manager):

A0:5, A1:4, A2:5,
B0:?, B1:?, B2:?,
C0:?, C1:?, C2:?

As the team leader of Team A, I have aggregated our team's total chocolate count. Team A has a total of 14 chocolates. It's time to find out Team B's total chocolate count.
NEXT: B0

--------------------------------------------------------------------------------
B0 (to chat_manager):

A0:5, A1:4, A2:5,
B0:3, B1:?, B2:?,
C0:?, C1:?, C2:?

As the team leader of Team B, I will start by sharing that I have 3 chocolates. Now, let's find out how many chocolates B1 has.
NEXT: B1

--------------------------------------------------------------------------------
B1 (to chat_manager):

A0:5, A1:4, A2:5,
B0:3, B1:1, B2:?,
C0:?, C1:?, C2:?

I have 1 chocolate. Let's find out how many chocolates B2 has.
NEXT: B2

--------------------------------------------------------------------------------
B2 (to chat_manager):

A0:5, A1:4, A2:5,
B0:3, B1:1, B2:2,
C0:?, C1:?, C2:?

I have 2 chocolates. Now, we need to report back to our team leader, B0, to aggregate our team's total chocolate count.
NEXT: B0

--------------------------------------------------------------------------------
B0 (to chat_manager):

A0:5, A1:4, A2:5,
B0:3, B1:1, B2:2,
C0:?, C1:?, C2:?

As the team leader of Team B, I have aggregated our team's total chocolate count. Team B has a total of 6 chocolates. It's time to find out Team C's total chocolate count.
NEXT: C0

--------------------------------------------------------------------------------
C0 (to chat_manager):

A0:5, A1:4, A2:5,
B0:3, B1:1, B2:2,
C0:1, C1:?, C2:?

As the team leader of Team C, I have 1 chocolate. Now, let's find out how many chocolates C1 has.
NEXT: C1

--------------------------------------------------------------------------------
C1 (to chat_manager):

A0:5, A1:4, A2:5,
B0:3, B1:1, B2:2,
C0:1, C1:4, C2:?

I have 4 chocolates. Let's find out how many chocolates C2 has.
NEXT: C2

--------------------------------------------------------------------------------
C2 (to chat_manager):

A0:5, A1:4, A2:5,
B0:3, B1:1, B2:2,
C0:1, C1:4, C2:5

I have 5 chocolates. Now, we need to report back to our team leader, C0, to aggregate our team's total chocolate count.
NEXT: C0

--------------------------------------------------------------------------------
C0 (to chat_manager):

A0:5, A1:4, A2:5,
B0:3, B1:1, B2:2,
C0:1, C1:4, C2:5

As the team leader of Team C, I have aggregated our team's total chocolate count. Team C has a total of 10 chocolates. Now that we have the total tally from all three teams, we can sum up all three teams' tally.

Team A: 14 chocolates
Team B: 6 chocolates
Team C: 10 chocolates

Total: 30 chocolates

TERMINATE

--------------------------------------------------------------------------------
Categories
API

Mistral Function Calling

pip install -U mistralai pandas yfinance rich
export MISTRAL_API_KEY=xxxxxxxxxxxx

Example 1: Stock Price

import pandas as pd
import functools, json
from rich import print as rich_print
import yfinance as yf
from mistralai.client import MistralClient
from mistralai.models.chat_completion import ChatMessage

model = "mistral-large-latest"
client = MistralClient()

# Updated function to get stock price
def get_stock_price(symbol: str) -> str:
    stock = yf.Ticker(symbol)
    hist = stock.history(period="1d")
    current_price = hist['Close'].iloc[0]  # Extract closing price
    print(f"Current price for {symbol}: {current_price}")
    return json.dumps({'price': current_price})

# Integrating the stock price function into tools
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_stock_price",
            "description": "Get the current stock price of a company",
            "parameters": {
                "type": "object",
                "properties": {
                    "symbol": {
                        "type": "string",
                        "description": "The stock symbol of the company.",
                    }
                },
                "required": ["symbol"],
            },
        },
    }
]

names_to_functions = {
    'get_stock_price': get_stock_price,
}

# Example user query
user_query = "What's the stock price of AAPL?"
print(f"User query: {user_query}")
messages = [
    ChatMessage(role="user", content=user_query)
]

response = client.chat(
    model=model,
    messages=messages,
    tools=tools,
    tool_choice="auto"
)
messages.append(response.choices[0].message)
rich_print(messages)

# Execute function
tool_call = response.choices[0].message.tool_calls[0]
function_name = tool_call.function.name
function_params = json.loads(tool_call.function.arguments)
print(f"\nExecuting function: {function_name} with parameters: {function_params}")

function_result = names_to_functions[function_name](**function_params)
print(f"Function result: {function_result}")

messages.append(ChatMessage(role="tool", name=function_name, content=function_result))

# Final model answer
response = client.chat(
    model=model,
    messages=messages
)
print(f"Final response: {response.choices[0].message.content}")
print("End of process.")
rich_print(response)
❯ python app.py
User query: What's the stock price of AAPL?
[
    ChatMessage(role='user', content="What's the stock price of AAPL?", name=None, tool_calls=None),
    ChatMessage(
        role='assistant',
        content='',
        name=None,
        tool_calls=[
            ToolCall(
                id='null',
                type=<ToolType.function: 'function'>,
                function=FunctionCall(name='get_stock_price', arguments='{"symbol": "AAPL"}')
            )
        ]
    )
]

Executing function: get_stock_price with parameters: {'symbol': 'AAPL'}
Current price for AAPL: 181.4199981689453
Function result: {"price": 181.4199981689453}
Final response: The stock price of AAPL is $181.42.
End of process.
ChatCompletionResponse(
    id='5f77a1aac50747b4b74d3a0e96907c1f',
    object='chat.completion',
    created=1709185946,
    model='mistral-large-latest',
    choices=[
        ChatCompletionResponseChoice(
            index=0,
            message=ChatMessage(role='assistant', content='The stock price of AAPL is $181.42.', name=None, tool_calls=[]),
            finish_reason=<FinishReason.stop: 'stop'>
        )
    ],
    usage=UsageInfo(prompt_tokens=79, total_tokens=95, completion_tokens=16)
)

Example 2

import pandas as pd
import functools, json
from rich import print as rich_print
from mistralai.client import MistralClient
from mistralai.models.chat_completion import ChatMessage

model = "mistral-large-latest"
client = MistralClient()

# Database
data = {
    'transaction_id': ['T1001', 'T1002', 'T1003', 'T1004', 'T1005'],
    'customer_id': ['C001', 'C002', 'C003', 'C002', 'C001'],
    'payment_amount': [125.50, 89.99, 120.00, 54.30, 210.20],
    'payment_date': ['2021-10-05', '2021-10-06', '2021-10-07', '2021-10-05', '2021-10-08'],
    'payment_status': ['Paid', 'Unpaid', 'Paid', 'Paid', 'Pending']
}
df = pd.DataFrame(data)
print("Database loaded successfully.")

# Retrieve payment status
def retrieve_payment_status(df: pd.DataFrame, transaction_id: str) -> str:
    if transaction_id in df.transaction_id.values:
        status = df[df.transaction_id == transaction_id].payment_status.item()
        print(f"Payment status for {transaction_id}: {status}")
        return json.dumps({'status': status})
    print(f"Error: Transaction ID {transaction_id} not found.")
    return json.dumps({'error': 'Transaction ID not found.'})

# Retrieve payment date
def retrieve_payment_date(df: pd.DataFrame, transaction_id: str) -> str:
    if transaction_id in df.transaction_id.values:
        date = df[df.transaction_id == transaction_id].payment_date.item()
        print(f"Payment date for {transaction_id}: {date}")
        return json.dumps({'date': date})
    print(f"Error: Transaction ID {transaction_id} not found.")
    return json.dumps({'error': 'Transaction ID not found.'})

tools = [
    {
        "type": "function",
        "function": {
            "name": "retrieve_payment_status",
            "description": "Get payment status of a transaction",
            "parameters": {
                "type": "object",
                "properties": {
                    "transaction_id": {
                        "type": "string",
                        "description": "The transaction id.",
                    }
                },
                "required": ["transaction_id"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "retrieve_payment_date",
            "description": "Get payment date of a transaction",
            "parameters": {
                "type": "object",
                "properties": {
                    "transaction_id": {
                        "type": "string",
                        "description": "The transaction id.",
                    }
                },
                "required": ["transaction_id"],
            },
        },
    }
]

names_to_functions = {
  'retrieve_payment_status': functools.partial(retrieve_payment_status, df=df),
  'retrieve_payment_date': functools.partial(retrieve_payment_date, df=df)
}

# User query
user_query = "What's the status of my transaction?"
print(f"User query: {user_query}")
messages = [
    ChatMessage(role="user", content=user_query)
]

# Model response
response = client.chat(
    model=model,
    messages=messages,
    tools=tools,
    tool_choice="auto"
)
print(f"Model response: {response.choices[0].message.content}")
messages.append(ChatMessage(role="assistant", content=response.choices[0].message.content))
transaction_details = "My transaction ID is T1001."
print(f"Transaction details provided: {transaction_details}")
messages.append(ChatMessage(role="user", content=transaction_details))

response = client.chat(
    model=model,
    messages=messages,
    tools=tools,
    tool_choice="auto"
)
messages.append(response.choices[0].message)

# Execute function
tool_call = response.choices[0].message.tool_calls[0]
function_name = tool_call.function.name
function_params = json.loads(tool_call.function.arguments)
print(f"\nExecuting function: {function_name} with parameters: {function_params}")

function_result = names_to_functions[function_name](**function_params)
print(f"Function result: {function_result}")

messages.append(ChatMessage(role="tool", name=function_name, content=function_result))

# Final model answer
response = client.chat(
    model=model,
    messages=messages
)
rich_print(messages)
print(f"Final response: {response.choices[0].message.content}")