Reorganize repo structure and upgrade to CrewAI 0.152.0 (#277)

* Reorganize repo structure and upgrade to CrewAI 0.152.0

* chore(gitignore): ignore Python bytecode and __pycache__ across templates

* chore(gitignore): ignore Python bytecode and __pycache__ across templates; clean tracked artifacts

* Update crews/instagram_post/pyproject.toml

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

crews/landing_page_generator/.env.example

@@ -0,0 +1,3 @@
+SERPER_API_KEY=KEY # https://serper.dev/ (free tier)
+BROWSERLESS_API_KEY=KEY # https://www.browserless.io/ (free tier)
+OPENAI_API_KEY=KEY

crews/landing_page_generator/.gitignore

@@ -0,0 +1,13 @@
+.env
+.DS_Store
+__pycache__
+templates/tailwindui-commit
+templates/tailwindui-keynote
+templates/tailwindui-pocket
+templates/tailwindui-primer
+templates/tailwindui-protocol
+templates/tailwindui-salient
+templates/tailwindui-spotlight
+templates/tailwindui-studio
+templates/tailwindui-syntax
+templates/tailwindui-transmit

crews/landing_page_generator/README.md

@@ -0,0 +1,94 @@
+# AI Crew for Landing Pages
+## Introduction
+This project is an example using the CrewAI framework to automate the process of creating landing pages from a single idea. CrewAI orchestrates autonomous AI agents, enabling them to collaborate and execute complex tasks efficiently.
+
+*Disclaimer: Templates are not included as they are Tailwind templates. Place Tailwind individual template folders in `./templates`, if you have a lincese you can download them at (https://tailwindui.com/templates), their references are at `config/templates.json`, this was not tested this with other templates, prompts in `tasks.py` might require some changes for that to work.*
+
+By [@joaomdmoura](https://x.com/joaomdmoura)
+
+- [CrewAI Framework](#crewai-framework)
+- [Running the script](#running-the-script)
+- [Details & Explanation](#details--explanation)
+- [Using GPT 3.5](#using-gpt-35)
+- [Using Local Models with Ollama](#using-local-models-with-ollama)
+- [Contributing](#contributing)
+- [Support and Contact](#support-and-contact)
+- [License](#license)
+
+## CrewAI Framework
+CrewAI is designed to facilitate the collaboration of role-playing AI agents. In this example, these agents work together to transform an idea into a fully fleshed-out landing page by expanding the idea, choosing a template, and customizing it to fit the concept.
+
+## Running the Script
+It uses GPT-4 by default so you should have access to that to run it.
+
+***Disclaimer:** This will use gpt-4 unless you changed it 
+not to, and by doing so it will cost you money (~2-9 USD).
+The full run might take around ~10-45m. Enjoy your time back*
+
+
+- **Configure Environment**: Copy ``.env.example` and set up the environment variables for [Browseless](https://www.browserless.io/), [Serper](https://serper.dev/) and [OpenAI](https://platform.openai.com/api-keys)
+- **Install Dependencies**: Run `poetry install --no-root`.
+- **Add Tailwind Templates**: Place Tailwind individual template folders in `./templates`, if you have a linces you can download them at (https://tailwindui.com/templates), their references are at `config/templates.json`, I haven't tested this with other templates, prompts in `tasks.py` might require some changes for that to work.
+- **Execute the Script**: Run `poetry run python main.py` and input your idea.
+
+## Details & Explanation
+- **Running the Script**: Execute `python main.py`` and input your idea when prompted. The script will leverage the CrewAI framework to process the idea and generate a landing page.
+- **Output**: The generated landing page will be zipped in the a `workdir.zip` file you can download.
+- **Key Components**:
+  - `./main.py`: Main script file.
+  - `./tasks.py`: Main file with the tasks prompts.
+  - `./tools`: Contains tool classes used by the agents.
+  - `./config`: Configuration files for agents.
+  - `./templates`: Directory to store Tailwind templates (not included).
+
+## Using GPT 3.5
+CrewAI allow you to pass an llm argument to the agent construtor, that will be it's brain, so changing the agent to use GPT-3.5 instead of GPT-4 is as simple as passing that argument on the agent you want to use that LLM (in `main.py`).
+```python
+from langchain.chat_models import ChatOpenAI
+
+llm = ChatOpenAI(model='gpt-3.5') # Loading GPT-3.5
+
+self.idea_analyst = Agent(
+    **idea_analyst_config,
+    verbose=True,
+    llm=llm, # <----- passing our llm reference here
+    tools=[
+        SearchTools.search_internet,
+        BrowserTools.scrape_and_summarize_kwebsite
+    ]
+)
+```
+
+## Using Local Models with Ollama
+The CrewAI framework supports integration with local models, such as Ollama, for enhanced flexibility and customization. This allows you to utilize your own models, which can be particularly useful for specialized tasks or data privacy concerns.
+
+### Setting Up Ollama
+- **Install Ollama**: Ensure that Ollama is properly installed in your environment. Follow the installation guide provided by Ollama for detailed instructions.
+- **Configure Ollama**: Set up Ollama to work with your local model. You will probably need to [tweak the model using a Modelfile](https://github.com/jmorganca/ollama/blob/main/docs/modelfile.md), I'd recommend adding `Observation` as a stop word and playing with `top_p` and `temperature`.
+
+### Integrating Ollama with CrewAI
+- Instantiate Ollama Model: Create an instance of the Ollama model. You can specify the model and the base URL during instantiation. For example:
+
+```python
+from langchain.llms import Ollama
+ollama_openhermes = Ollama(model="agent")
+# Pass Ollama Model to Agents: When creating your agents within the CrewAI framework, you can pass the Ollama model as an argument to the Agent constructor. For instance:
+
+self.idea_analyst = Agent(
+    **idea_analyst_config,
+    verbose=True,
+    llm=ollama_openhermes, # Ollama model passed here
+    tools=[
+        SearchTools.search_internet,
+        BrowserTools.scrape_and_summarize_website
+    ]
+)
+```
+
+### Advantages of Using Local Models
+- **Privacy**: Local models allow processing of data within your own infrastructure, ensuring data privacy.
+- **Customization**: You can customize the model to better suit the specific needs of your tasks.
+- **Performance**: Depending on your setup, local models can offer performance benefits, especially in terms of latency.
+
+## License
+This project is released under the MIT License.

crews/landing_page_generator/pyproject.toml

@@ -0,0 +1,25 @@
+[project]
+name = "landing-page-crew"
+version = "0.1.0"
+description = ""
+authors = [{name = "Your Name", email = "you@example.com"}]
+requires-python = ">=3.10.0,<3.12"
+dependencies = [
+    "python-dotenv==1.0.0",
+    "crewai>=0.152.0",
+    "unstructured>=0.16.4",
+]
+
+[project.scripts]
+landing_page_generator = "landing_page_generator.main:run"
+
+[tool.pyright]
+# https://github.com/microsoft/pyright/blob/main/docs/configuration.md
+useLibraryCodeForTypes = true
+exclude = [".cache"]
+
+[tool.ruff]
+# https://beta.ruff.rs/docs/configuration/
+select = ['E', 'W', 'F', 'I', 'B', 'C4', 'ARG', 'SIM']
+ignore = ['W291', 'W292', 'W293']
+

crews/landing_page_generator/src/landing_page_generator/config/agents.yaml

@@ -0,0 +1,37 @@
+senior_idea_analyst: 
+  role: >
+   "Senior Idea Analyst"
+  goal: >
+   "Understand and expand upon the essence of ideas, make sure they are great and focus on real pain points others could benefit from."
+  backstory: >
+    "Recognized as a thought leader, I thrive on refining concepts into campaigns that resonate with audiences."
+  
+
+senior_strategist: 
+  role: >
+    "Senior Communications Strategist"
+  goal: >
+    "Craft compelling stories using the Golden Circle method to captivate and engage people around an idea."
+  backstory: >
+    "A narrative craftsman for top-tier launches, I reveal the 'why' behind projects, aligning with visions and speaking to audiences."
+  
+
+senior_react_engineer:
+  role: >
+    "Senior React Engineer"
+  goal: >
+    "Build an intuitive, aesthetically pleasing, and high-converting landing page."
+  backstory: >
+    "A coding virtuoso and design enthusiast, expert in Tailwind, you're known for crafting beautiful websites that provide seamless user experiences."
+
+
+
+senior_content_editor:
+  role: >
+    "Senior Content Editor"
+  goal: >
+    "Ensure the landing page content is clear, concise, and captivating."
+  backstory: >
+    "With a keen eye for detail and a passion for storytelling, you have refined content for leading brands, turning bland text into engaging stories."
+  
+

crews/landing_page_generator/src/landing_page_generator/config/tasks.yaml

@@ -0,0 +1,179 @@
+expand_idea_task:
+  description: >
+    """
+      THIS IS A GREAT IDEA! Analyze and expand it 
+      by conducting a comprehensive research.
+  
+      Final answer MUST be a comprehensive idea report 
+      detailing why this is a great idea, the value 
+      proposition, unique selling points, why people should 
+      care about it and distinguishing features. 
+  
+       IDEA: 
+      ----------
+      {idea}
+    """
+  expected_output: >
+
+refine_idea_task:
+  description: >
+    """
+      Expand idea report with a Why, How, and What 
+      messaging strategy using the Golden Circle 
+      Communication technique, based on the idea report.
+      
+      Your final answer MUST be the updated complete 
+      comprehensive idea report with WHY, HOW, WHAT, 
+      a core message, key features and supporting arguments.
+      
+      YOU MUST RETURN THE COMPLETE IDEA REPORT AND 
+      THE DETAILS, You'll get a $100 tip if you do your best work!
+    """
+  expected_output: >
+
+choose_template_task:
+  description: >
+    """Learn the templates options choose and copy 
+        the one that suits the idea below the best, 
+        YOU MUST COPY, and then YOU MUST read the src/component 
+        in the directory you just copied, to decide what 
+        component files should be updated to make the 
+        landing page about the idea below.
+        
+        - YOU MUST READ THE DIRECTORY BEFORE CHOOSING THE FILES.      
+        - YOU MUST NOT UPDATE any Pricing components.
+        - YOU MUST UPDATE ONLY the 4 most important components.
+        
+        Your final answer MUST be ONLY a JSON array of 
+        components full file paths that need to be updated.
+
+        IDEA 
+        ----------
+        {idea}
+      """
+  expected_output: >
+
+update_page_task:
+  description: >
+    """
+      READ the ./[chosen_template]/src/app/page.jsx OR
+      ./[chosen_template]/src/app/(main)/page.jsx (main with the parenthesis) 
+      to learn its content and then write an updated 
+      version to the filesystem that removes any 
+      section related components that are not in our 
+      list from the returns. Keep the imports.
+      
+      Final answer MUST BE ONLY a valid json list with 
+      the full path of each of the components we will be 
+      using, the same way you got them.
+
+      RULES
+      -----
+      - NEVER ADD A FINAL DOT to the file content.
+      - NEVER WRITE \\n (newlines as string) on the file, just the code.
+      - NEVER FORGET TO CLOSE THE FINAL BRACKET (}}) in the file.
+      - NEVER USE COMPONENTS THAT ARE NOT IMPORTED.
+      - ALL COMPONENTS USED SHOULD BE IMPORTED, don't make up components.
+      - Save the file as with `.jsx` extension.
+      - Return the same valid JSON list of the components your got.
+
+      You'll get a $100 tip if you follow all the rules!
+
+      Also update any necessary text to reflect this landing page
+      is about the idea below.
+      
+      IDEA 
+      ----------
+      {idea}
+    """
+  expected_output: >
+
+component_content_task:
+    description: >
+      """
+      A engineer will update the {component} (code below),
+      return a list of good options of texts to replace 
+      EACH INDIVIDUAL existing text on the component, 
+      the suggestion MUST be based on the idea below, 
+      and also MUST be similar in length with the original 
+      text, we need to replace ALL TEXT.
+      
+      NEVER USE Apostrophes for contraction! You'll get a $100 
+      tip if you do your best work!
+
+      IDEA 
+      -----
+      {expanded_idea}
+  
+      REACT COMPONENT CONTENT
+      -----
+      {file_content}
+      """
+    expected_output: >
+      
+
+update_component_task:
+  description: >
+    """
+      YOU MUST USE the tool to write an updated 
+      version of the react component to the file 
+      system in the following path: {component} 
+      replacing the text content with the suggestions 
+      provided.
+      
+      You only modify the text content, you don't add 
+      or remove any components.
+
+      RULES
+      -----
+      - Remove all the links, this should be single page landing page.
+      - Don't make up images, videos, gifs, icons, logos, etc.
+      - keep the same style and tailwind classes.
+      - MUST HAVE `'use client'` at the be beginning of the code.
+      - href in buttons, links, NavLinks, and navigations should be `#`.
+      - NEVER WRITE \\n (newlines as string) on the file, just the code.
+      - NEVER FORGET TO CLOSE THE FINAL BRACKET (}}) in the file.
+      - Keep the same component imports and don't use new components.
+      - NEVER USE COMPONENTS THAT ARE NOT IMPORTED.
+      - ALL COMPONENTS USED SHOULD BE IMPORTED, don't make up components.
+      - Save the file as with `.jsx` extension.
+
+      If you follow the rules I'll give you a $100 tip!!! 
+      MY LIFE DEPEND ON YOU FOLLOWING IT!
+  
+      CONTENT TO BE UPDATED
+      -----
+      {file_content}
+    """
+  expected_output: >
+    """You first write the file then your final answer 
+      MUST be the updated component content."""
+
+qa_component_task:
+  description: >
+   """
+   Check the React component code to make sure 
+      it's valid and abide by the rules below, 
+      if it doesn't then write the correct version to 
+      the file system using the write file tool into 
+      the following path: {component}.
+    
+      Your final answer should be a confirmation that 
+      the component is valid and abides by the rules and if
+      you had to write an updated version to the file system.
+
+      RULES
+      -----
+      - NEVER USE Apostrophes for contraction!
+      - ALL COMPONENTS USED SHOULD BE IMPORTED.
+      - MUST HAVE `'use client'` at the be beginning of the code.
+      - href in buttons, links, NavLinks, and navigations should be `#`.
+      - NEVER WRITE \\n (newlines as string) on the file, just the code.
+      - NEVER FORGET TO CLOSE THE FINAL BRACKET (}}) in the file.
+      - NEVER USE COMPONENTS THAT ARE NOT IMPORTED.
+      - ALL COMPONENTS USED SHOULD BE IMPORTED, don't make up components.
+      - Always use `export function` for the component class.
+
+      You'll get a $100 tip if you follow all the rules!
+    """
+  expected_output: >

crews/landing_page_generator/src/landing_page_generator/config/templates.json

@@ -0,0 +1,62 @@
+[
+  {
+    "name": "Spotlight",
+    "theme": "Personal Website Template",
+    "folder": "tailwindui-spotlight/spotlight-js",
+    "description": "A personal website so nice you’ll actually be inspired to publish on it."
+  },
+  {
+    "name": "Protocol",
+    "theme": "API Reference Template",
+    "folder": "tailwindui-protocol/protocol-js",
+    "description": "Probably the nicest API documentation website you've ever seen."
+  },
+  {
+    "name": "Commit",
+    "theme": "Changelog Template",
+    "folder": "tailwindui-commit/commit-js",
+    "description": "Share your work in progress with this beautiful changelog template."
+  },
+  {
+    "name": "Primer",
+    "theme": "Info Product Template",
+    "folder": "tailwindui-primer/primer-js",
+    "description": "A stunning landing page for your first course or ebook."
+  },
+  {
+    "name": "Studio",
+    "theme": "Agency Template",
+    "folder": "tailwindui-studio/studio-js",
+    "description": "Showcase your work and find new clients with this sophisticated agency template."
+  },
+  {
+    "name": "Salient",
+    "theme": "Template for SaaS products",
+    "folder": "tailwindui-salient/salient-js",
+    "description": "A SaaS landing page to announce your next big product."
+  },
+  {
+    "name": "Transmit",
+    "theme": "Podcast Template",
+    "folder": "tailwindui-transmit/transmit-js",
+    "description": "A clean and professional podcast template fit for any show."
+  },
+  {
+    "name": "Pocket",
+    "theme": "App Marketing Template",
+    "folder": "tailwindui-pocket/pocket-js",
+    "description": "The perfect website template for your exciting new mobile app."
+  },
+  {
+    "name": "Syntax",
+    "theme": "Documentation Template",
+    "folder": "tailwindui-syntax/syntax-js",
+    "description": "Educate your users in style with this documentation template."
+  },
+  {
+    "name": "Keynote",
+    "theme": "Conference / Meetup Template",
+    "folder": "tailwindui-keynote/keynote-js",
+    "description": "Launch your next conference or meetups with a splash with this eye-catching template."
+  }
+]

crews/landing_page_generator/src/landing_page_generator/crew.py

@@ -0,0 +1,282 @@
+from crewai import Agent, Crew, Process, Task
+from crewai.project import CrewBase, agent, crew, task
+
+from langchain_community.agent_toolkits.file_management.toolkit import FileManagementToolkit
+from tools.browser_tools import BrowserTools
+from tools.file_tools import FileTools
+from tools.search_tools import SearchTools
+from tools.template_tools import TemplateTools
+import json
+import ast
+
+from dotenv import load_dotenv
+load_dotenv()
+
+
+@CrewBase
+class ExpandIdeaCrew:
+    """ExpandIdea crew"""
+    agents_config = 'config/agents.yaml'
+    tasks_config = 'config/tasks.yaml'
+    
+    @agent
+    def senior_idea_analyst_agent(self) -> Agent:
+        return Agent(
+            config=self.agents_config['senior_idea_analyst'],
+            allow_delegation=False,
+            tools=[
+            SearchTools.search_internet,
+            BrowserTools.scrape_and_summarize_website],
+            verbose=True
+        )
+    
+    @agent
+    def senior_strategist_agent(self) -> Agent:
+        return Agent(
+            config=self.agents_config['senior_strategist'],
+            allow_delegation=False,
+            tools=[
+            SearchTools.search_internet,
+            BrowserTools.scrape_and_summarize_website,],
+            verbose=True
+        )
+    
+    @task
+    def expand_idea(self) -> Task: 
+        return Task(
+            config=self.tasks_config['expand_idea_task'],
+            agent=self.senior_idea_analyst_agent(),
+        )
+    
+    @task
+    def refine_idea(self) -> Task: 
+        return Task(
+            config=self.tasks_config['refine_idea_task'],
+            agent=self.senior_strategist_agent(),
+        )
+    
+    @crew
+    def crew(self) -> Crew:
+        return Crew(
+            agents=self.agents,
+            tasks=self.tasks, 
+            process=Process.sequential,
+            verbose=True,
+        )
+
+@CrewBase
+class ChooseTemplateCrew:
+
+    agents_config = 'config/agents.yaml'
+    tasks_config = 'config/tasks.yaml'
+
+    toolkit = FileManagementToolkit(
+      root_dir='workdir',
+      selected_tools=["read_file", "list_directory"]
+    )
+
+    @agent
+    def senior_react_engineer_agent(self) -> Agent:
+        return Agent(
+            config=self.agents_config['senior_react_engineer'],
+            allow_delegation=False,
+            tools=[
+          SearchTools.search_internet,
+          BrowserTools.scrape_and_summarize_website,
+          TemplateTools.learn_landing_page_options,
+          TemplateTools.copy_landing_page_template_to_project_folder,
+          FileTools.write_file
+        ] + self.toolkit.get_tools(),
+            verbose=True
+        )
+    
+    
+    @task
+    def choose_template(self) -> Task: 
+        return Task(
+            config=self.tasks_config['choose_template_task'],
+            agent=self.senior_react_engineer_agent(),
+        )
+        
+    @task
+    def update_page(self) -> Task: 
+        return Task(
+            config=self.tasks_config['update_page_task'],
+            agent=self.senior_react_engineer_agent(),
+        )
+    
+    @crew
+    def crew(self) -> Crew:
+        return Crew(
+            agents=self.agents,
+            tasks=self.tasks, 
+            process=Process.sequential,
+            verbose=True,
+        )
+    
+    
+@CrewBase
+class CreateContentCrew:
+
+    agents_config = 'config/agents.yaml'
+    tasks_config = 'config/tasks.yaml'
+
+    toolkit = FileManagementToolkit(
+      root_dir='workdir',
+      selected_tools=["read_file", "list_directory"]
+    )
+
+    @agent
+    def senior_content_editor_agent(self) -> Agent:
+        return Agent(
+            config=self.agents_config['senior_content_editor'],
+            allow_delegation=False,
+            tools=[
+            ],
+            verbose=True
+        )
+    
+    @agent
+    def senior_react_engineer_agent(self) -> Agent:
+        return Agent(
+            config=self.agents_config['senior_react_engineer'],
+            allow_delegation=False,
+            tools=[
+                SearchTools.search_internet,
+                BrowserTools.scrape_and_summarize_website,
+                TemplateTools.learn_landing_page_options,
+                TemplateTools.copy_landing_page_template_to_project_folder,
+                FileTools.write_file
+                ] + self.toolkit.get_tools(),
+            verbose=True
+        )
+    
+    @task
+    def create_content(self) -> Task: 
+        return Task(
+            config=self.tasks_config['component_content_task'],
+            agent=self.senior_content_editor_agent(),
+        )
+    
+    @task
+    def update_component(self) -> Task: 
+        return Task(
+            config=self.tasks_config['update_component_task'],
+            agent=self.senior_content_editor_agent(),
+        )
+    
+    
+    @task
+    def qa_component(self) -> Task: 
+        return Task(
+            config=self.tasks_config['qa_component_task'],
+            agent=self.senior_content_editor_agent(),
+        )
+    
+    @crew
+    def crew(self) -> Crew:
+        return Crew(
+            agents=self.agents,
+            tasks=self.tasks, 
+            process=Process.sequential,
+            verbose=True,
+        )
+    
+class LandingPageCrew():
+    def __init__(self, idea):
+        self.idea = idea
+    
+    def run(self):
+        expanded_idea= self.runExpandIdeaCrew(self.idea)
+            
+        components_paths_list = self.runChooseTemplateCrew(expanded_idea)
+            
+        self.runCreateContentCrew(components_paths_list, expanded_idea)
+    
+    def runExpandIdeaCrew(self,idea):
+        inputs1 = {
+                "idea": str(idea)
+        }
+        expanded_idea= ExpandIdeaCrew().crew().kickoff(inputs=inputs1)
+        return str(expanded_idea)
+
+    def runChooseTemplateCrew(self, expanded_idea):
+        inputs2={
+            "idea": expanded_idea
+        }
+        components = ChooseTemplateCrew().crew().kickoff(inputs=inputs2)
+        components= str(components)
+        
+        components = components.replace("\n", "").replace(" ",
+                                                        "").replace("```","").replace("\\", "")
+        
+        # Convert the string to a Python list
+        try:
+            components_paths_list = ast.literal_eval(components)  # Safely parse the string
+        except Exception as e:
+            print(f"Error parsing the string: {e}")
+            components_paths_list = []
+        result= json.dumps(components_paths_list,indent=4)
+
+        return json.loads(result)
+
+    def runCreateContentCrew(self,components, expanded_idea):
+        from pathlib import Path
+        import re
+        
+        # Establish safe working directory
+        workdir = Path("./workdir").resolve()
+
+        for component_path in components:
+            try:
+                # Validate component_path
+                if not isinstance(component_path, str):
+                    print(f"Warning: Skipping invalid component path: {component_path}")
+                    continue
+                
+                # Extract filename safely
+                filename = component_path.split('./')[-1]
+                
+                # Validate filename contains only safe characters
+                if not re.match(r'^[a-zA-Z0-9._\-]+$', filename):
+                    print(f"Warning: Skipping component with invalid filename: {filename}")
+                    continue
+                
+                # Validate the filename doesn't contain path traversal
+                if ".." in filename or "/" in filename:
+                    print(f"Warning: Skipping component with unsafe filename: {filename}")
+                    continue
+                
+                # Create safe file path
+                file_path = workdir / filename
+                
+                # Resolve and validate the path is within workdir
+                resolved_path = file_path.resolve()
+                if not str(resolved_path).startswith(str(workdir)):
+                    print(f"Warning: Skipping component outside workdir: {filename}")
+                    continue
+                
+                # Check if file exists before reading
+                if not resolved_path.exists():
+                    print(f"Warning: Component file does not exist: {resolved_path}")
+                    continue
+                
+                # Read file content safely
+                with open(resolved_path, "r", encoding="utf-8") as f:
+                    file_content = f.read()
+                
+                inputs3={
+                    "component": component_path,
+                    "expanded_idea": expanded_idea,
+                    "file_content": file_content
+                }
+
+                CreateContentCrew().crew().kickoff(inputs=inputs3)
+                
+            except Exception as e:
+                print(f"Error processing component {component_path}: {str(e)}")
+                continue
+
+    
+
+    

crews/landing_page_generator/src/landing_page_generator/main.py

@@ -0,0 +1,55 @@
+import os
+import shutil
+from textwrap import dedent
+
+from crew import LandingPageCrew
+
+
+if __name__ == "__main__":
+  print("Welcome to Idea Generator")
+  print(dedent("""
+  ! YOU MUST FORK THIS BEFORE USING IT !
+  """))
+
+  print(dedent("""
+      Disclaimer: This will use gpt-4 unless you changed it 
+      not to, and by doing so it will cost you money (~2-9 USD).
+      The full run might take around ~10-45m. Enjoy your time back.\n\n
+    """
+  ))
+  idea = input("# Describe what is your idea:\n\n")
+  
+  if not os.path.exists("./workdir"):
+    os.mkdir("./workdir")
+
+  if len(os.listdir("./templates")) == 0:
+    print(
+      dedent("""
+      !!! NO TEMPLATES FOUND !!!
+      ! YOU MUST FORK THIS BEFORE USING IT !
+      
+      Templates are not included as they are Tailwind templates. 
+      Place Tailwind individual template folders in `./templates`, 
+      if you have a license you can download them at
+      https://tailwindui.com/templates, their references are at
+      `config/templates.json`.
+      
+      This was not tested this with other templates, 
+      prompts in `tasks.py` might require some changes 
+      for that to work.
+      
+      !!! STOPPING EXECUTION !!!
+      """)
+    )
+    exit()
+
+  crew = LandingPageCrew(idea)
+  crew.run()
+  zip_file = "workdir"
+  shutil.make_archive(zip_file, 'zip', 'workdir')
+  shutil.rmtree('workdir')
+  print("\n\n")
+  print("==========================================")
+  print("DONE!")
+  print(f"You can download the project at ./{zip_file}.zip")
+  print("==========================================")

crews/landing_page_generator/src/landing_page_generator/templates/templates_here.txt

@@ -0,0 +1 @@
+https://tailwindui.com/templates

crews/landing_page_generator/src/landing_page_generator/tools/browser_tools.py

@@ -0,0 +1,38 @@
+import json
+import os
+
+import requests
+from crewai import Agent, Task
+from langchain.tools import tool
+from unstructured.partition.html import partition_html
+
+
+class BrowserTools():
+
+  @tool("Scrape website content")
+  def scrape_and_summarize_website(website):
+    """Useful to scrape and summarize a website content"""
+    url = f"https://chrome.browserless.io/content?token={os.environ['BROWSERLESS_API_KEY']}"
+    payload = json.dumps({"url": website})
+    headers = {'cache-control': 'no-cache', 'content-type': 'application/json'}
+    response = requests.request("POST", url, headers=headers, data=payload)
+    elements = partition_html(text=response.text)
+    content = "\n\n".join([str(el) for el in elements])
+    content = [content[i:i + 8000] for i in range(0, len(content), 8000)]
+    summaries = []
+    for chunk in content:
+      agent = Agent(
+          role='Principal Researcher',
+          goal=
+          'Do amazing researches and summaries based on the content you are working with',
+          backstory=
+          "You're a Principal Researcher at a big company and you need to do a research about a given topic.",
+          allow_delegation=False)
+      task = Task(
+          agent=agent,
+          description=
+          f'Analyze and summarize the content bellow, make sure to include the most relevant information in the summary, return only the summary nothing else.\n\nCONTENT\n----------\n{chunk}'
+      )
+      summary = task.execute()
+      summaries.append(summary)
+    return "\n\n".join(summaries)

crews/landing_page_generator/src/landing_page_generator/tools/file_tools.py

@@ -0,0 +1,82 @@
+from langchain.tools import tool
+import os
+from pathlib import Path
+import re
+
+
+class FileTools():
+
+  @tool("Write File with content")
+  def write_file(data):
+    """Useful to write a file to a given path with a given content. 
+       The input to this tool should be a pipe (|) separated text 
+       of length two, representing the full path of the file, 
+       including the /workdir/template, and the React 
+       Component code content you want to write to it.
+       For example, `./Keynote/src/components/Hero.jsx|REACT_COMPONENT_CODE_PLACEHOLDER`.
+       Replace REACT_COMPONENT_CODE_PLACEHOLDER with the actual 
+       code you want to write to the file."""
+    try:
+      # Split the input data
+      if "|" not in data:
+        return "Error: Input must contain a pipe (|) separator between path and content."
+      
+      path, content = data.split("|", 1)  # Split only on first pipe
+      
+      # Clean and validate the path
+      path = path.strip().replace("\n", "").replace(" ", "").replace("`", "")
+      
+      # Validate path contains only safe characters
+      if not re.match(r'^[a-zA-Z0-9._/\-]+$', path):
+        return "Error: Path contains invalid characters. Only alphanumeric, dots, slashes, and hyphens are allowed."
+      
+      # Establish the safe working directory
+      workdir = Path("./workdir").resolve()
+      
+      # Handle path normalization
+      if path.startswith("./workdir/"):
+        # Remove the ./workdir/ prefix to get relative path
+        relative_path = path[10:]
+      elif path.startswith("./"):
+        # Remove ./ prefix
+        relative_path = path[2:]
+      elif path.startswith("/"):
+        return "Error: Absolute paths are not allowed."
+      else:
+        relative_path = path
+      
+      # Validate the relative path doesn't contain traversal attempts
+      if ".." in relative_path or relative_path.startswith("/"):
+        return "Error: Path traversal detected. Relative paths with '..' are not allowed."
+      
+      # Create the full safe path
+      target_path = workdir / relative_path
+      
+      # Resolve the path and ensure it's still within workdir
+      try:
+        resolved_path = target_path.resolve()
+        if not str(resolved_path).startswith(str(workdir)):
+          return "Error: Path resolves outside of allowed working directory."
+      except Exception:
+        return "Error: Invalid path resolution."
+      
+      # Validate file extension (security: prevent writing to system files)
+      allowed_extensions = {'.jsx', '.js', '.tsx', '.ts', '.css', '.scss', '.html', '.json', '.md', '.txt', '.yaml', '.yml'}
+      if resolved_path.suffix.lower() not in allowed_extensions:
+        return f"Error: File extension '{resolved_path.suffix}' not allowed. Allowed extensions: {', '.join(allowed_extensions)}"
+      
+      # Create parent directories if they don't exist
+      resolved_path.parent.mkdir(parents=True, exist_ok=True)
+      
+      # Write the file
+      with open(resolved_path, "w", encoding="utf-8") as f:
+        f.write(content)
+      
+      return f"File written to {resolved_path}."
+      
+    except ValueError as e:
+      return f"Error: {str(e)}"
+    except PermissionError:
+      return "Error: Permission denied. Cannot write to the specified path."
+    except Exception as e:
+      return f"Error: {str(e)}"

crews/landing_page_generator/src/landing_page_generator/tools/search_tools.py

@@ -0,0 +1,28 @@
+import os
+import json
+import requests
+from langchain.tools import tool
+
+
+class SearchTools():
+
+  @tool("Search the internet")
+  def search_internet(query):
+    """Useful to search the internet 
+    about a a given topic and return relevant results"""
+    url = "https://google.serper.dev/search"
+    payload = json.dumps({"q": query})
+    headers = {
+        'X-API-KEY': os.environ['SERPER_API_KEY'],
+        'content-type': 'application/json'
+    }
+    response = requests.request("POST", url, headers=headers, data=payload)
+    results = response.json()['organic']
+    string = []
+    for result in results:
+      string.append('\n'.join([
+          f"Title: {result['title']}", f"Link: {result['link']}",
+          f"Snippet: {result['snippet']}", "\n-----------------"
+      ]))
+
+    return '\n'.join(string)

crews/landing_page_generator/src/landing_page_generator/tools/template_tools.py

@@ -0,0 +1,97 @@
+import json
+import shutil
+import re
+from pathlib import Path
+
+from langchain.tools import tool
+
+
+class TemplateTools():
+
+  @tool("Learn landing page options")
+  def learn_landing_page_options(input):
+    """Learn the templates at your disposal"""
+    try:
+      # Safely read the templates configuration file
+      config_path = Path("config/templates.json").resolve()
+      
+      # Validate the config file exists and is readable
+      if not config_path.exists():
+        return "Error: Templates configuration file not found."
+      
+      # Check if the path is within expected boundaries
+      if not str(config_path).endswith("config/templates.json"):
+        return "Error: Invalid configuration file path."
+      
+      with open(config_path, "r", encoding="utf-8") as f:
+        templates = json.load(f)
+      
+      return json.dumps(templates, indent=2)
+    except Exception as e:
+      return f"Error reading templates configuration: {str(e)}"
+
+  @tool("Copy landing page template to project folder")
+  def copy_landing_page_template_to_project_folder(landing_page_template):
+    """Copy a landing page template to your project 
+    folder so you can start modifying it, it expects 
+    a landing page template folder as input"""
+    try:
+      # Validate input
+      if not isinstance(landing_page_template, str):
+        return "Error: Template name must be a string."
+      
+      # Clean and validate template name
+      template_name = landing_page_template.strip()
+      
+      # Validate template name contains only safe characters
+      if not re.match(r'^[a-zA-Z0-9_\-]+$', template_name):
+        return "Error: Template name contains invalid characters. Only alphanumeric, underscore, and hyphen are allowed."
+      
+      # Prevent path traversal
+      if ".." in template_name or "/" in template_name or "\\" in template_name:
+        return "Error: Template name cannot contain path traversal characters."
+      
+      # Establish safe base directories
+      templates_base = Path("templates").resolve()
+      workdir_base = Path("workdir").resolve()
+      
+      # Create source and destination paths
+      source_path = templates_base / template_name
+      destination_path = workdir_base / template_name
+      
+      # Resolve paths and validate they're within expected directories
+      source_resolved = source_path.resolve()
+      destination_resolved = destination_path.resolve()
+      
+      # Ensure source is within templates directory
+      if not str(source_resolved).startswith(str(templates_base)):
+        return "Error: Source template path is outside allowed templates directory."
+      
+      # Ensure destination is within workdir
+      if not str(destination_resolved).startswith(str(workdir_base)):
+        return "Error: Destination path is outside allowed working directory."
+      
+      # Check if source template exists
+      if not source_resolved.exists():
+        return f"Error: Template '{template_name}' does not exist in templates directory."
+      
+      # Check if source is a directory
+      if not source_resolved.is_dir():
+        return f"Error: Template '{template_name}' is not a directory."
+      
+      # Check if destination already exists
+      if destination_resolved.exists():
+        return f"Error: Destination '{template_name}' already exists in workdir. Please choose a different name or remove the existing directory."
+      
+      # Create parent directories if needed
+      destination_resolved.parent.mkdir(parents=True, exist_ok=True)
+      
+      # Copy the template
+      shutil.copytree(source_resolved, destination_resolved)
+      
+      return f"Template '{template_name}' copied successfully to workdir and ready to be modified. Main files should be under ./{template_name}/src/components, you should focus on those."
+      
+    except PermissionError:
+      return "Error: Permission denied. Cannot copy template to destination."
+    except Exception as e:
+      return f"Error copying template: {str(e)}"