Introduction

An AI agent is a sophisticated software entity designed to perform tasks autonomously or semi-autonomously by perceiving its environment, making decisions, and taking actions to achieve specific goals. These agents leverage various artificial intelligence techniques, including machine learning, natural language processing, and computer vision, to interpret data and respond to dynamic situations. AI agents can operate in diverse domains, from virtual personal assistants like Siri and Alexa to more complex systems in robotics, healthcare, and finance. Their ability to learn from experience and adapt to new information makes them invaluable tools in modern technology.

AI agents excel at tackling complex problems that are often too intricate or time-consuming for human beings to solve efficiently. By processing vast amounts of data at incredible speeds, they can identify patterns, generate insights, and propose solutions that might not be immediately apparent. For instance, in healthcare, AI agents can analyze patient records to predict disease outbreaks or recommend personalized treatment plans. In finance, they can assess market trends and manage investment portfolios with precision. Their ability to continuously learn and improve enhances their problem-solving capabilities, making them essential partners in navigating the complexities of today's world.

AI agents powered by large language models (LLMs) can significantly enhance the processes of researching and writing blogs on various topics. These agents are designed to autonomously perform tasks such as gathering information, analyzing data, and generating coherent text. By leveraging their ability to understand and generate human language, LLM agents can sift through vast amounts of online content, extract relevant insights, and summarize findings effectively. This capability enables them to help writers pinpoint key themes and trends in their chosen subjects, making the research phase of content creation more efficient.

This blog explores the development of an AI agent capable of conducting in-depth research and generating blog content. We will utilize CrewAI as a framework for this agent and the gpt-3.5-turbo-0125 model as the LLM.

What is CrewAI?

CrewAI is an open-source multi-agent orchestration framework designed to facilitate the collaboration of autonomous AI agents. This Python-based platform allows agents to work together as a cohesive unit, or "crew," to accomplish complex tasks efficiently. Each agent within CrewAI assumes a specific role, enabling them to delegate tasks, communicate, and utilize both existing and custom tools to achieve their goals. This framework aims to simplify the automation of multi-agent workflows, making it accessible for developers and researchers seeking to harness the power of collaborative AI.

Utilizing CrewAI for building AI agents offers several advantages that enhance the development and deployment of intelligent systems. Its modular design allows for easy customization and integration of various tools, enabling developers to create agents that can perform a wide range of tasks. The framework excels in facilitating seamless communication among agents, which is crucial for collaborative problem-solving and efficient task management. Additionally, CrewAI supports the use of open-source large language models (LLMs), allowing agents to leverage advanced reasoning capabilities. This combination of flexibility, collaboration, and powerful AI tools positions CrewAI as a robust solution for creating sophisticated AI agents that can tackle real-world challenges effectively.

CrewAI Core Concepts

1. Agents: These are independent units designed to perform tasks, make decisions, and communicate with other agents. They can use tools, ranging from simple search functions to complex integrations with other systems, APIs, etc.

2. Tasks: Tasks are jobs or assignments that an AI agent needs to complete. They can also include details like which agent should do it and what tools they might need.

3. Crew: A crew is a team of agents, each with a specific role, working together to achieve a common goal. Forming a crew involves gathering agents, defining their tasks, and setting up the order in which tasks will be done.

AI Agent Code Walkthrough

This post will explore two methodologies for AI-driven blog creation. The first method relies solely on the LLM's internal knowledge base. The second approach involves interacting with external tools to gather additional information on the specified topic, which is then used to generate blog content.

Installing Dependencies

For this project, the following dependencies should be installed.

pip install crewai crewai-tools

Defining Agents

Three specialized AI agents have been developed: a Research Analyst, a Content Writer, and an Editor. As the names imply, the Research Analyst is tasked with gathering and synthesizing information on a given topic. Subsequently, the Content Writer leverages this research output to produce engaging blog content. Finally, the Editor refines the generated content, ensuring grammatical accuracy, coherence, and overall quality.

researcher = Agent(
   role='Research Analyst',
   goal="Provide an indepth research report on a single or list of {topic}",
   backstory='An expert researcher with a keen eye for details on a topic.',
   verbose=True
)

writer = Agent(
   role='Content Writer',
   goal="Craft engaging blog posts about {topic}",
   backstory='A skilled writer with a passion for technology.',
   verbose=True
)

editor = Agent(
   role='Editor',
   goal='Edit and refine the content produced by the writer',
   backstory='An experienced editor with a knack for polishing content.',
   verbose=True
)

Let's try to understand each of the attributes we have used in the Agent definition:

1. role: Defines the agent's function within the crew. It determines the kind of tasks the agent is best suited for.

2. goal: The individual objective that the agent aims to achieve. It guides the agent's decision-making process.

3. backstory: Provides context to the agent's role and goal, enriching the interaction and collaboration dynamics.

4. verbose: Setting this to True configures the internal logger to provide detailed execution logs, aiding in debugging and monitoring. The default is False.

Beyond the four primary agent attributes outlined, a comprehensive set of additional attributes is available to fine-tune agent behavior. A complete list of these agent attributes can be accessed here.

The agent goal value in the above Agent code example incorporates a topic placeholder for dynamic input. This allows for flexible topic assignment during Crew execution.

Defining Tasks

Three distinct tasks have been delineated for assignment to specialized agents. The initial task, encompassing comprehensive research on a designated topic, is delegated to the Research Analyst agent. Subsequently, the Content Writer agent is entrusted with the creation of blog content based on the research findings. To ensure optimal output, the Editor agent is assigned the responsibility of meticulous review and refinement of the generated blog content.

research = Task(
   description="Break down the subject and do indepth research about {topic}. Explain the functionality, use cases, etc. of the topic",
   expected_output="An elaborate summary of 4000 words on the topic or list of topics: {topic}",
   agent=researcher
)

write = Task(
   description="Write an exhaustive engaging blog post about the {topic}, based on the research analyst’s summary.",
   expected_output='A 2000 word blog post formatted in markdown with engaging, informative, and accessible content, avoiding complex jargon.',
   agent=writer,
)

editing = Task(
   description='Edit and refine the blog post written by the Content Writer.',
   expected_output="A polished blog, free of grammatical errors, with coherent structure and engaging language. "
       "Making sure the blog is easy to read and understand.",
   agent=editor,
   output_file='blog-posts/leadership_lessons.md' # The final blog post will be saved here
)

Let's first go through the task attributes:

1. description: A clear, concise statement of what the task entails.

2. expected_output: A detailed description of what the task's completion looks like.

3. agent: The agent responsible for the task, assigned either directly or by the crew's process.

4. output_file: Saves the task output to a file.

Here is the comprehensive list of all the task attributes provided by the framework.

Let's examine each task in detail.

The first task involves the Researcher agent, which is responsible for conducting an in-depth analysis of the subject by breaking it down into smaller components. The goal is for the agent to thoroughly understand the topic, learn its functionalities and use cases, and then explain them in simplified terms. For instance, if tasked with researching "What qualities make a good leader," the agent should first identify these qualities and then clarify how they contribute to effective leadership. The expected outcome of this task is a comprehensive summary of 4000 words on the topic.

The second task is assigned to the Content Writer agent. This agent's role is to synthesize the information from the research report and craft an engaging blog post. The blog should be written in simple English to ensure accessibility, while still incorporating all essential information from the research. The target output for this task is a 2000-word blog formatted in markdown.

The final task is designated to the Editor agent. The Editor's responsibility is to review and refine the blog post before it is published. This involves correcting any grammatical errors and ensuring that the blog's structure is coherent. After making the final edits, the Editor will save the content in a markdown file.

Defining Crew

Next, we need to integrate the Agents and their respective Tasks, for that we would need to define a crew. A crew is a coordinated group of agents working collectively to complete a series of tasks. Each crew outlines the strategy for task execution, agent collaboration, and the overall workflow.

crew = Crew(
   agents=[researcher, writer, editor],
   tasks=[research, write, editing],
   verbose=True,
   process=Process.sequential,
)

Let's first understand each attribute of the Crew:

1. agents: A list of agents that are part of the crew.

2. tasks: A list of tasks assigned to the crew.

3. verbose: The verbosity level for logging during execution.

4. process: The process flow (e.g., sequential, hierarchical) the crew follows.

Having defined the constituent agents and their respective tasks, we now focus on process orchestration. CrewAI currently supports two primary process types: sequential and hierarchical. In a sequential process the tasks are executed one after another, allowing for a linear flow of work. In a hierarchical process, the manager agent coordinates the crew, delegating tasks and validating outcomes before proceeding.

Executing Crew

Once the crew is assembled, we can initiate the workflow with the kickoff() method. This starts the execution process according to the defined process flow.

post_inputs = {
   'topic': "What Makes a Great Leader? The Impact of Leadership on Team Success. "
   "Core Qualities of an Inspirational Leader. Leader creating a Culture of Encouragement and Support. "
   "Empowering Others to Lead."
}

# Execute tasks
crew.kickoff(inputs=post_inputs)

The kickoff function accepts a post_inputs dictionary as an argument, which includes the desired blog topic. To optimize blog generation, it is recommended to provide a detailed topic description incorporating subheadings. This approach enhances control over the final output. For example, instead of a generic topic like What makes a Good leader, specify subheadings such as What Makes a Great Leader? The Impact of Leadership on Team Success, and so forth.

Response

The blog generated by the Crew is shown below. It's important to note that the content is based solely on the LLM's internal knowledge base. Consequently, the blog's accuracy is contingent upon the comprehensiveness and currency of the LLM's training data. This dependence exposes the potential for inaccuracies due to outdated or incomplete information within the LLM's dataset.

Generating a blog using external tools

To solve the problem mentioned above we can use these two tools provided by the CrewAI framework:

1. SerperDevTool

2. WebsiteSearchTool

The SerperDevTool allows the agent to search the internet with a search_query. The WebsiteSearchTool allows the Agent to semantic search a query from a specific URL content. These two tools can be utilized by the Research Analyst Agent to gather information from the internet and conduct research on the latest data.

The tools are defined below.

from crewai_tools import (
   SerperDevTool,
   WebsiteSearchTool
)

os.environ["SERPER_API_KEY"] = "<YOUR_SERPER_API_KEY>"

# Instantiate tools
search_tool = SerperDevTool()
web_rag_tool = WebsiteSearchTool()

To use the SerperDevTool you need to sign up with Serper. You can sign up to Serper here and generate the API key.

Below is the updated code of the researcher agent where we have set these 2 tools.

researcher = Agent(
   role='Research Analyst',
   goal="Provide an indepth research report on a single or list of {topic}",
   backstory='An expert researcher with a keen eye for details on a topic.',
   tools=[search_tool, web_rag_tool],
   verbose=True
)

With these changes, we'll rerun our Crew to generate the blog. The content produced by the crew is shown below.

We can see that the content quality of our blog has significantly improved with the use of these tools. The complete code for the application is provided below.

import os
from crewai import Agent, Task, Crew, Process


from crewai_tools import (
   SerperDevTool,
   WebsiteSearchTool
)


os.environ["SERPER_API_KEY"] = "<YOUR_SERPER_API_KEY>"
os.environ["OPENAI_MODEL_NAME"] = 'gpt-3.5-turbo-0125'


# Instantiate tools
search_tool = SerperDevTool()
web_rag_tool = WebsiteSearchTool()


# Create agents
researcher = Agent(
   role='Research Analyst',
   goal="Provide an indepth research report on a single or list of {topic}",
   backstory='An expert researcher with a keen eye for details on a topic.',
   tools=[search_tool, web_rag_tool],
   verbose=True
)


writer = Agent(
   role='Content Writer',
   goal="Craft engaging blog posts about {topic}",
   backstory='A skilled writer with a passion for technology.',
   verbose=True
)


editor = Agent(
   role='Editor',
   goal='Edit and refine the content produced by the writer',
   backstory='An experienced editor with a knack for polishing content.',
   verbose=True
)


# Define tasks
research = Task(
   description="Break down the subject and do indepth research about {topic}. Explain the functionality, use cases, etc. of the topic",
   expected_output="An elaborate summary of 4000 words on the topic or list of topics: {topic}",
   agent=researcher
)


write = Task(
   description="Write an exhaustive engaging blog post about the {topic}, based on the research analyst’s summary.",
   expected_output='A 2000 word blog post formatted in markdown with engaging, informative, and accessible content, avoiding complex jargon.',
   agent=writer,
)


editing = Task(
   description='Edit and refine the blog post written by the Content Writer.',
   expected_output="A polished blog, free of grammatical errors, with coherent structure and engaging language. "
       "Making sure the blog is easy to read and understand.",
   agent=editor,
   output_file='blog-posts/leadership_lessons_with_plan.md' # The final blog post will be saved here
)


crew = Crew(
   agents=[researcher, writer, editor],
   tasks=[research, write, editing],
   verbose=True,
   process=Process.sequential,
)


post_inputs = {
   'topic': "What Makes a Great Leader? The Impact of Leadership on Team Success. "
   "Core Qualities of an Inspirational Leader. Leader creating a Culture of Encouragement and Support. "
   "Empowering Others to Lead."
}


# Execute tasks
crew.kickoff(inputs=post_inputs)

Conclusion

This blog has explored the construction of a multi-agent system using CrewAI. We delved into CrewAI's core components, dissected the architecture of a multi-agent application, and demonstrated its execution. By leveraging CrewAI's tools, we learned to harness external information for research purposes. While this blog provides a foundational understanding, the potential applications of multi-agent systems are vast. If you have any questions regarding the subject, please don't hesitate to ask in the comments section. I will be more than happy to address them.

I frequently create content on building LLM applications, AI agents, vector databases, and other RAG-related topics. If you're interested in similar articles, consider subscribing to my blog.

If you're in the Generative AI space or LLM application domain, let's connect on Linkedin! I'd love to stay connected and continue the conversation. Reach me at: linkedin.com/in/ritobrotoseth