Consumers, designers, prosumers and creators are experiencing an unprecedented level of control and automation, enabled by AI. A new category of companies is emerging to enable this shift.
Why now?
The barriers to new product development continue to decline.
AI-driven interactions are blurring the lines between creator/designer and engineer/developer.
Foundation model costs are declining, opening up opportunities to serve long-tail users first.
We’re at the early innings of this shift, and so with Context Aware, we aim to:
Observe trends, teams and companies shaping these new interactions.
Surface opportunities for consumers, creators, designers and prosumers who would like to use these products or join these teams.
This Week’s Context: Betting on the Browser
Almost every online interaction has been designed for a human to process and understand, navigate and manipulate. That’s changing quickly - we aren’t the only ones “seeing” or navigating the internet anymore.
In early 2024, WebVoyager demonstrated that LLMs have the ability to combine visual context (like screenshots) with textual context. Agents can use multimodal cues to manipulate a browser — in other words, models can “see” online interfaces and therefore trigger automations and actions based on that visual data. Browsers that were originally designed with humans in mind, don’t need to be
A number of companies have begun to leverage this insight to build web agents. Anthropic’s* Computer Use was released in October 2024. In January, OpenAI launched Operator, which uses this insight to automate web-based tasks using natural language. Agent performance on browser-based tasks is still far from perfect — a NYTimes article highlights the advancements but also shortcomings of OpenAI’s Operator product today — but companies are starting to demonstrate meaningful performance improvements. Just in the past 2 months, numerous companies have demonstrated 80%+ performance on the WebVoyager benchmark, vs the original project’s performance of 59.1%. We will likely see continued improvement in performance as companies continue to iterate and launch new web agent products. Adoption will likely follow the improving performance/accuracy and speed of these tasks.
Where will the interesting opportunities emerge?
New interfaces. Web page and software application interfaces may change, as fewer tasks require humans to log into those systems. Developers and designers will likely need to think about how both humans and agents will navigate their product, and how it will be searched by agent-based search engines. Rotem Weiss, CEO and co-founder of Tavily, which allows developers to connect their apps to the web, proposes a need for “standard protocols and communication frameworks” to create agent interoperability.
Business model disruption. We could see an impact on ad-based models or marketplaces that today are premised on humans rather than bots searching.
Co-pilots vs. agents. Co-pilot driven features could become more powerful for tasks that are not automated entirely by AI. Interfaces may be able to give users real-time guidance and/or recommendations while they are working. (One example is the Gemini Cursor GitHub project, which demonstrates the ability to guide a user on an Amazon page).
Verticalized agents. Companies may seek to verticalize subsets of both consumer and enterprise tasks that require browser-based or multimodal understanding. Some tasks (like filing taxes, or inputting data into legacy/complex systems) will likely require verticalization of this technology.
Developer ecosystem. Companies like Perplexity*, Airtop and Tavily are already enabling developers to incorporate web data into their products via API. An ecosystem of infrastructure providers that let developers build more sophisticated agents will emerge.
Recent Product Launches
An overview of product launches from the past few weeks that are innovating on interactions & interfaces for consumers, creators and prosumers.
Deta.surf has built integrated AI functionality into the browsing experience in ways that feel deeply personalized. Users are able to search for key moments within YouTube videos without leaving the page, or highlight important content and request visualizations from them on the spot.
RPLY. We have all had messages buried deep in our inbox, often left unread, or when read, we respond with an “I’m so sorry!!” Announced last week, RPLY is an AI-powered iMessage assistant designed to address this issue. By scanning your iMessages, RPLY identifies unanswered texts and suggests AI-generated responses.
Alma.food. Alma enables effortless tracking of your daily food intake and nutrients—not by manually logging each item, but through intuitive multimodal inputs like snapping a photo or using voice commands.
A0.dev. A0.dev generates custom React Native apps based on a prompt - app makers are able share directly without the pain of connecting to Testflight.
Replit launch of mobile app. Last week, Replit launched their agent on mobile, meaning you can create an app on your phone with no code, on the go. This feature pushes us closer to a world where people are able to make one off, hyper-specific apps for their personalized needs.
Onlook describes themselves as “Cursor for Designers.” Their product allows you to dynamically change your visual interface, while automatically updating your code. It’s the go-to tool for thousands of design engineers, and for good reason.
Hugging Face Spaces allows users to easily search and find 400k+ AI apps, as well as create and deploy ML-powered demos super quickly.
Fundraising & Acquisitions
Indigo is bringing AI to real estate transactions. The company announced an $8M fundraise led by NFX to streamline the home buying & negotiation process.
ShopMy.* Millions of Americans work full-time as content creators. ShopMy is enabling this segment of entrepreneurs by more seamlessly connecting brands and creators together. One of the core components of ShopMy’s platform is their shelves, which creators can build and share with users who are looking for a more targeted shopping experience. ShopMy just raised $77.5M led by Bessemer Venture Partners.
Tana.inc raises $25M from Tola Capital. Tana supercharges the note-taking experience by infusing AI-driven automations. Tana sees an opportunity to use note-taking as the launching point for automation by connecting notes to a multitude of other SaaS tools.
Looking to hire
Lore is a consumer AI-powered platform that reimagines how people explore and engage with their obsessions—fandoms, niche history, internet culture, memes, and more. A pre-seed VC-backed startup, they’re building a place where you can do all your internet rabbit-holing in one place—for the endlessly obsessed. To do this, they’re looking for strong people to join the NYC-based team
Founding Engineer (Web + AI)
If you’re a full-stack engineer or backend-focused engineer who loves search, AI, and consumer products, this is an opportunity to shape something new from the ground up.
Founding Vibes Curator (Socials, Brand, Messaging)
SWSH is the easiest way to share photo albums. They’re looking to hire:
Mainframe is building the future of interfaces. They’re looking to hire exceptional engineers to help them do this.
*Bessemer portfolio company
Social Media Disclaimer