Overview

The W&B Agent is in active development and in private preview. Contact W&B Support to request access to this feature.

W&B Agent is your personalized AI research assistant helping you analyze experiments and understand results. The W&B Agent can help you investigate your project’s data, find patterns across runs, recommend next steps, build visualizations, reports and more.

W&B Agent can assist with common tasks:

• Analyze your experiments to find patterns and insights.
• Propose next steps for your experiments, such as recommending hyperparameters to try.
• Create new plot and panels to visualize your data.
• Build saved views and reports to share insights with your team.

Chat with the agent in your project’s workspace.

1. From any project, click W&B Agent at the upper right corner of the page.
   
2. In the modal, enter your question or request, then click the Send button, which has an upward-pointing arrow.

Depending on the complexity of your question, the agent might take a few moments to respond. You can close the modal while the agent is working and return to check on the agent’s response later.

While the agent works, you might see Reasoning and tool steps:

• Reasoning steps summarize how the agent is approaching your question.
• Tool steps show when the agent uses a tool, such as creating a panel, filtering runs, creating Python scripts, and so forth.

Click on the Reasoning or Tool step to expand more details about the agent’s thought process or the specific tool action it took, respectively.

Start a new chat

To start a new chat, open the W&B Agent, then click Start a new chat (pencil icon).

Starting a new chat does not delete your previous conversations. To view previous conversations, open the Chat history tab.

View chat history

To view previous conversations with the W&B Agent:

1. Open the W&B Agent.
2. Click Chat history.
3. Select a conversation.

Undock chat

You can undock the chat into a separate window, which allows you to view the agent’s response side-by-side with your project data.

To undock the chat, click the undock icon in the upper right corner of the chat window next to the close (X) button.

Provide feedback

Click the thumbs up or thumbs down icon at the end of the response to provide feedback on W&B Agent’s response. Your feedback helps W&B improve the W&B Agent’s performance and accuracy.

The queries you make, and the agent’s responses to those queries within the chat session, are shared to W&B. You can optionally provide additional context about your feedback in the text box that appears after you select thumbs up or thumbs down.

For information about how W&B handles data usage for the W&B Agent, see Governance and security.

Example chats

The following are examples of real chats with the W&B Agent to illustrate how you can interact with the agent and the types of responses you can expect from the agent.

Gain insights on your experiments

Suppose you are an ML Engineer who has just completed a series of runs for a new model. You want to understand how your model performed, identify potential issues, and decide on your next steps.

To do this, you navigate to your the project where you logged runs. You open the W&B Agent and ask:

User prompt

What patterns do you see across my runs?

The following is an example of how the W&B Agent might respond to this question. First, the agent makes reasoning steps to determine how to approach the question:

W&B Agent reasoning steps

I’m considering running a state breakdown scan with counts by state using
specific server filters. I’ll look at the array of states: ['finished',
'running', 'crashed', 'failed', 'killed', 'pending', 'preempting'] and
determine the length of each. Summing these metrics could provide
useful insights. I should also focus on identifying objective metrics
from sample keys while excluding those that start with "_" or "system/".
It’s important to compute the best min/max for numeric summary metrics too.

The agent identifies that it needs to analyze the data across runs to answer the question. To do this, the agent creates a Python script (analyze_patterns.py) that uses the W&B Python SDK to query previously logged runs to identify patterns in the data. Once the script is generated, the agent executes it within a sandbox environment.

Next, the agent uses the ouput to generate a summary of the patterns it found across the runs, which it shares in the chat.

The agent was able to identify that my current experiment does not log validation/test metrics and recommended logging those metrics in future runs to better understand the model’s performance.

As a next step, you can ask the agent to help set up an experiment with validation metrics logged. To do this, you can ask the agent:

User prompt

Can you help me update my Python script to also log validation metrics?

The agent can then generate the code to log validation metrics in your training script, which you can copy and paste into your codebase.

Get recommendations for hyperparameter tuning

Suppose you are an ML Engineer who has just completed a series of runs for a new model. You want to understand how your model performed and get recommendations for how to improve your model’s performance in future runs. You decide to ask your W&B Agent for recommendations on hyperparameter tuning.

You might ask the agent:

User prompt

I want to conduct a sweep to improve lower my training model's loss. Do you have suggestions on what hyperparameters I should try?

The agent might respond with the following reasoning steps:

W&B Agent reasoning steps

 The user is looking for insights to improve training loss, so I
 should suggest ranges for learning rate, optimizer, batch size,
 weight decay, and other relevant parameters.

Since the user wants to conduct a sweep, I need to show the code or
instrumentation with a clear yet concise YAML or Python configuration.
I might also suggest using random or Bayesian techniques for more
effective hyperparameter selection.

The agent identifies that it needs to analyze the hyperparameters used in previous runs and their relationship to the training loss to provide recommendations for hyperparameter tuning.

The agent identifies that the given project used a learning rate of 0.01 for all runs. Based on these insights, the agent recommends trying a lower learning rate, such as 0.001, in future runs to see if that helps improve the model’s performance.

The agent also identifies that of the runs that use the SGD optimizer. Based on this insight, the agent recommends trying a different optimizer, such as Adam (Adaptive Moment Estimation), in future runs to see if that helps improve the model’s performance.

The following image shows a portion of the agent’s response with its recommendations for hyperparameter tuning:

Create a report to share with my team

Suppose you are a data scientist who has been analyzing your project’s runs and identified interesting patterns in the data. You want to share these insights with your team in an easy-to-digest format. You decide to ask your W&B Agent to help you create a W&B Report.

To do this, you open the W&B Agent and ask:

User prompt

Create a report to share my findings with my team.

The agent might respond with the following reasoning steps:

W&B Agent reasoning steps

I’m looking to query the recent 50 runs, summarizing key frequencies 
and counts by using api.runs to slice them down. I'll be counting
lengths with lazy evaluation and weaving together the stats
for a proper report.  I need to include a description that might
require importing different report libraries.

The agent identifies that it needs to create a W&B Report that summarizes the insights it found in the data. As part of it’s reasoning process, it generates a Python script that uses the W&B SDK to create a report with the relevant data and insights. The agent executes this script in a sandbox environment, which creates a W&B Report in your project.

The following image shows the how the agent creates a W&B Report in the sandbox environment and then returns the link to the report in the chat:

Under the hood, the W&B Agent creates a W&B Report using the Workspaces and Reports API.