Extract Posts by Topic from X to JSON

This template automates searching X.com (formerly Twitter) for a specific topic or keyword. It then extracts the content and engagement metrics (likes, retweets, replies) for a specified number of tweets, providing a structured dataset for analysis.

Back to all templates

Airtop Community

Featured Apps

Tags

This automation streamlines the process of gathering data from X.com for research and analysis. Here's how it works:

Starts on X.com: The automation begins by opening a browser window and navigating to https://x.com.
Checks Login Status: It verifies that you are logged into your X.com account. If not, it will stop and prompt you to log in.
Performs a Search: Using the topic you provide (e.g., "AI Agents"), it automatically enters the query into the search bar and executes the search.
Extracts Tweet Data: Once the search results load, the automation begins to extract key information from each tweet, including the full text, and the number of likes, retweets, and replies.
Scrolls for More: To gather the desired number of tweets, it intelligently scrolls down the page, loading and extracting more results.
Delivers Results: After collecting the specified number of tweets or exhausting the search results, the automation compiles all the data and presents it in a structured JSON format, ready for you to use.

Usage Ideas

Market Research: Gather public opinion and sentiment about a new product, feature, or brand by searching for relevant keywords.
Competitive Analysis: Monitor what people are saying about your competitors to understand their strengths and weaknesses.
Content Strategy: Find popular tweets and trending topics within your niche to inspire your own content calendar.
Lead Generation: Identify potential customers who are talking about a problem that your product or service can solve.
Brand Monitoring: Keep track of mentions of your brand, products, or key personnel to manage your online reputation.

Customization Ideas

This template provides a solid foundation for X.com data extraction that you can easily adapt to your specific needs. You can ask the Agent Builder Assistant to:

Change the Search Criteria: Easily switch the topic or keyword for each run.
Adjust Data Volume: Specify exactly how many tweets you want to collect, from a handful to hundreds.
Customize Extracted Data: Modify the automation to capture different information from each tweet, such as the author's username, the date it was posted, or a direct link to the tweet.
Save Your Data: Add steps to automatically send the extracted tweet data to a Google Sheet, an Airtable base, or another destination for easy access and analysis.
Alter the Search Scope: Change the automation to extract tweets from a specific user's profile instead of a general topic search.

FROM MANUAL TO AUTOMATED IN MINUTES

Related Automations

Save hours of manual research while giving your team a steady stream of trending content ideas. This agent automatically monitors key LinkedIn thought leaders, analyzes their latest posts, and identifies emerging topics gaining traction in your industry. Each week, you’ll receive a concise email summarizing the top five trends, why they resonate, and how your team can use them to shape timely blog posts, campaigns, and social content that stay ahead of the conversation.

Airtop Community

Draft Meeting Follow-up Emails (Fathom)

Automatically analyzes meeting transcripts from the Fathom API and uses AI to generate concise, professional follow-up email drafts, complete with action items and next steps.

Amir Ashkenazi

Respond to Tweets with Human in the Loop

This automation monitors X.com (formerly Twitter) for new tweets matching your specified keyword, extracts detailed information about each tweet, generates a draft response using AI, and saves all the data to a Google Sheet.

Cesar Sanchez