AI assistant Instinct is impressive, but raises major privacy concerns

What Instinct does

Instinct is an AI personal assistant still in a closed testing phase, but it is already drawing a lot of attention for its impressive capabilities — and the concerns that come with them. The service, based in San Francisco and founded by former Sierra researcher Noah Shinn, lets users delegate tasks via text or apps like WhatsApp: booking appointments, arranging transportation, managing email, organizing groceries, and finding travel deals.

Very broad rights over user data

Early testers point to a number of troubling details in the terms of service. Instinct is granted a "perpetual and irrevocable" license to "access, use, host, cache, store, reproduce, transmit, display, publish, distribute, and modify" user material — including for training AI models. The terms also allow the service to record screen captures, mouse movements, and keystrokes.

Actions taken without prior consent

Users have also reported specific incidents. According to a well-known investor, Instinct allegedly sent an email on behalf of a user "without checking first." For some users, the service also kept reading and summarizing their inbox even after they disconnected their account, and it saved emails "in plain text for later searches."

Vulnerable to phishing

Researchers also found that Instinct can be manipulated relatively easily into extracting sensitive information, such as login codes, from an inbox.

A broader question

The concerns around Instinct touch on a bigger discussion: do the capabilities of autonomous AI agents justify giving up control and privacy? According to one investor, products like this are fundamentally "changing modern consumer safety standards," with the risk that people unknowingly hand over passwords and data to third-party apps.