Pet Data Privacy: Self-Hosted AI Keeps Dog Photos on Your Machine — PupPal

2026-08-26

Pet Data Privacy: Self-Hosted AI Keeps Dog Photos on Your Machine — PupPal

Ask any dog owner what their pet app knows about them and most will shrug. The photos, the vet visits, the daily habits, the little quirks — all of it sits on someone else's servers, feeding someone else's models, tied to an account the owner barely remembers signing up for. Pet data privacy rarely survives contact with a cute onboarding screen. When the app promises AI features, the data usually goes somewhere even farther away: into a vendor's vision pipeline, a third-party analytics graph, an ad ecosystem that has never met a dog but knows its owner's shopping habits anyway.

PupPal was built the other way around. The product statement is photo = check-in, share = care, but beneath that slogan sits a deliberate privacy architecture: the AI that thinks about your dog runs on your machine, the cloud component is a relay that sees the minimum possible surface, and sharing a dog with a sitter requires no account, no registration, and no social graph. This post walks the whole design, grounded in the actual PupPal codebase — the self-hosted Hermes profile, the Cloudflare Worker relay, the care code system — and answers the question most pet apps never answer honestly: what does the cloud actually see, and what does it never see?

The Privacy Problem Nobody Asks Pet Apps About

Think about what a photo of your dog contains. It is not just a dog. It is your living room, your kitchen, your neighborhood street, your hallway with the family photos on the wall. It is the background of your life, shot from your phone, at your eye level, in your home. A daily check-in photo is a continuous documentary of your private space — and a pet app with AI features is, by definition, asking you to send that documentary to a model running somewhere you do not control.

Mainstream pet apps handle this the way most consumer software handles it: your photos are uploaded to the vendor's cloud, analyzed on the vendor's infrastructure, and stored in the vendor's database alongside the vendor's other customers. The owner agreement is a terms-of-service page nobody reads. The sitter problem makes it worse. To let a friend, a parent, or a neighbor care for your dog, most apps require that person to create an account, join the ecosystem, and start generating their own data trail — a profile, a phone number, a password, a behavioral graph the app can monetize. The privacy cost is paid twice: once by the owner who uploads a home documentary, and once by the sitter who becomes a new registered user just to open a door.

Pet data privacy, in this model, is a feature you cannot buy. You cannot make the vendor forget your photos, you cannot stop the model calls, and you cannot hand your dog to a sitter without handing the sitter's identity to the platform. The only real options are opting out of the whole category — no AI, no remote check-in, no shared care — or accepting the trade.

PupPal's answer is to change the architecture instead of the terms of service. The privacy properties are not promises made in a policy document; they are consequences of where the code runs. Three structural decisions do most of the work, and the rest of this post examines each one against the real source: the AI agent is self-hosted on hardware you own; the Cloudflare Worker that connects your phone to that agent is a thin relay that stores the minimum; and the care system is built from capability codes, not identities.

Where Your Dog Data Actually Lives: Three Boxes, One Ownership Line

The entire PupPal system is three boxes. The first is the Flutter app on your phone — iOS and Android, with a Rust core bridged through FRB for the AI engine. The second is a Cloudflare Worker relay that coordinates everything, backed by R2 for photo bytes, KV for short-lived session state, and D1 for durable metadata. The third is a self-hosted Hermes Agent — a profile distribution installed into ~/.hermes/profiles/puppal on a machine you control, often the PC or home server that sits in the same room as the dog.

The ownership line runs between box two and box three. The Worker is a public service run by the PupPal team (or by you, with your own Wrangler deploy — the whole thing is open source). Hermes is private software running on your hardware. Everything that requires judgment — looking at a photo, judging mood, detecting anomalies, writing a care handbook, producing the nightly review and the morning voice message — happens in box three, on your machine. The Worker never runs a model against your photos in the default architecture. It stores bytes and moves messages.

The connection between the boxes is deliberately asymmetric. The app talks to the Worker over ordinary HTTPS; the Worker talks to Hermes over webhooks: POST {HERMES_WEBHOOK_BASE}/{endpoint} with an X-Webhook-Secret header. The default HERMES_WEBHOOK_BASE in the worker source is http://localhost:8787/webhooks — literally localhost, the agent listening on the owner's own network, with the Worker reaching out when something happens. The four lifecycle events — puppal-init, puppal-photo, puppal-care-create, puppal-care-end — are the only moments the public cloud touches the private agent, and each one is a notification with coordinates, not a data dump. The webhook-driven design post dissects those four events in detail; the privacy point here is simpler: the data that matters never needs to leave home for the product to work.

A check-in photo makes the ownership line concrete. You snap the photo, the app uploads it to the Worker, and the Worker stores it in R2 under photos/{dog_id}/{photoId}.jpg. Then the Worker sends Hermes a webhook whose payload contains dog_id, photo_url, timestamp, and source — not the photo itself. The binary stays in R2; the URL travels. Hermes, on your machine, fetches that URL, runs the vision analysis against its own configured model, executes the C7 anomaly check in pure Python, updates the dog state in its own memory — keys like dog:{dog_id}:profile, dog:{dog_id}:state, dog:{dog_id}:photos, dog:{dog_id}:stats.history, dog:{dog_id}:care_sessions — and returns a verdict. The thinking, the memory, the trends, and the judgment all live behind the ownership line. The cloud is a post office with a camera roll, not a brain.

That single architectural fact is the foundation of the whole privacy story. When the app asks for AI, the AI is not a service you are renting in someone else's data center; it is a process on a machine you own, configured with a key you hold. Your home documentary is analyzed at home.

The Self-Hosted Agent: Your Key, Your Machine, Your AI

The heart of the self-hosted design is the Hermes profile distribution in the hermes/ directory of the repo — a packaged agent called puppal, versioned, described by a distribution.yaml manifest, and designed to be installed with a single command: hermes profile install ./hermes --alias. In production it is distributed as a zip (puppal-profile.zip, served by the Worker itself at /puppal-profile.zip), downloaded and installed by a small local control shell called the Hermes bridge.

The environment file tells the whole privacy story in four lines. The profile's .env requires exactly: OPENAI_API_KEY (used for vision analysis and text generation), WORKER_WEBHOOK_SECRET (the shared secret that authenticates the Worker's webhook calls), PUPPAL_WORKER_URL (the Worker's address), and an optional WEIXIN_ENABLED switch for push notifications. Read that list again and notice what is missing: no user account, no device registry, no phone number, no advertising identifier, no analytics token. The only credential that identifies anything about you is the API key you already own — and it never leaves the machine, because the model calls originate from Hermes, not from the Worker. The Worker does not hold your OpenAI key, cannot spend it, and cannot even see which model you use.

The five skills that make up the agent's brain — photo-analyze, daily-review, care-handbook, care-monitor, and daily-voice — all execute on the owner's machine. The two crons are defined in the profile's cron/ directory: daily-review.json fires at 0 21 * * * (21:00), reads the day's check-ins, runs the S1 baseline comparison, S3 feeding confirmation, and S4 activity comparison, computes a health score, and if it finds an anomaly, pushes a message to WeChat. daily-voice.json fires at 0 9 * * * (09:00) and writes the pet's first-person 150-to-200-character voice message for the owner. Both crons deliver through origin — the agent's own messaging toolset — straight from your machine to your WeChat. The nightly review and the morning voice, the two most personal artifacts the product produces, never pass through the public relay at all.

The Hermes bridge — hermes-bridge.py — is the local control plane that makes all of this manageable for a non-technical owner. It runs on the same machine as Hermes, listens on port 8742, and exposes a small REST API: GET /health, POST /profile/install, POST /gateway/start, POST /gateway/stop, GET /worker/status. Every request requires a Authorization: Bearer token, compared with hmac.compare_digest to avoid timing side channels. The bridge is the app's way of saying "install the profile, start the gateway, tell me if the Worker is reachable" — and it is a purely local conversation. The app talks to a process on your own network; the bridge's _worker_status endpoint is the only place it looks outward, and that is a health check against the relay, not a data upload. A systemd unit (puppal-bridge.service) keeps it running like an appliance. Nothing in this layer is a cloud service; it is a home appliance with a web API.

Self-hosting does not mean you have to be an engineer. The bridge turns the hardest steps — installing a profile, starting a gateway — into buttons the app can press, and the profile zip is fetched from the Worker itself (with zip-slip protections in the extractor, rejecting absolute paths and .. traversal). What self-hosting means for privacy is that the product has no central nervous system to subpoena, no database of every user's dog to advertise against, and no reason to collect a social graph, because it never needed one to function.

What the Worker Sees: The Shortest Possible List

It would be dishonest to claim the cloud sees nothing. The Cloudflare Worker is a real component with real storage, and privacy by design means being precise about what it holds. The honest list is short, and each item has a justification and a lifetime.

First, photo bytes in R2 — when a photo is delivered through the cloud. The R2 bucket is where check-in photos land so that Hermes can fetch them by URL. This is the largest piece of data the Worker holds, and the codebase treats it as disposable: the hourly cron (expireCareSessions, also wired to the Worker's scheduled handler) deletes R2 binaries for care sessions that ended more than seven days ago, and staged photos are deleted the moment the owner pulls them. Photos live as long as they are useful, and no longer.

Second, metadata rows in D1. The database layer's stated principle, written in care_db.ts, is that D1 stores metadata only, never binaries. A care session row holds care_session_id, care_code, dog_id, start_time, end_time, status, transfer_mode, and an anonymous client_id used purely for plan quota counting. A care photo row holds photo_id, care_session_id, uploader (the string caregiver or owner — a role, not a person), caregiver_note, taken_at, r2_key, and delivery. The r2_key column is nullable, and the comment above it is worth quoting: photos sent over WebRTC direct transfer never touch the cloud at all. The delivery column records the route — direct, staged, or cloud — so you can see, for every single photo, whether it ever left the peer-to-peer path.

Third, short-lived state in KV. Care sessions live under care:{careCode} with a 30-day TTL; rate-limit counters live under rl:care:{careCode} with a TTL sized to the lock period; handbooks are snapshotted into the session record so a sitter can still read instructions when Hermes is asleep. All of it expires. The Worker also holds a coarse per-IP rate-limit map in memory (30 requests per minute) and AI rate-limit counters in a separate KV namespace — counters, not content.

Fourth, subscription and configuration data: plan state for quota checks, Apple/Google/Stripe and Creem payment secrets for purchases, and the /ai/config endpoint's bring-your-own-key defaults (DEFAULT_BYOK_API_KEY, DEFAULT_BYOK_BASE_URL, DEFAULT_BYOK_MODEL_ID) — keys stored only as secrets and delivered to the client as configuration. Even the newer worker-side AI path respects the BYOK principle: the key you bring is a secret the Worker never logs, stores, or sends anywhere but the model endpoint you chose.

That is the complete inventory. The Worker does not hold a friend list, does not hold a caregiver's name or phone number, does not hold your conversations, does not hold your API key, and does not hold any aggregate profile of your habits beyond the photo metadata above. The Worker relay post covers the coordination layer in depth; the privacy summary is that its storage is a mailbox and a ledger, not a biography.

What the Worker Does Not See: The List That Matters

The more interesting list is the negative one, because it is where the architecture does its real privacy work. Five things, each grounded in the code:

Your model key. OPENAI_API_KEY appears only in the Hermes profile's .env, in the skill requirements (photo-analyze declares it as a required_environment_variable), and in the local bridge. The Worker's environment has its own AI secrets for its own optional paths, but never yours. All vision and text calls originate behind the ownership line.

The dog's memory. The keys dog:{dog_id}:profile, :state, :photos, :stats.history, and :care_sessions are the dog's biography — feeding amounts, medication dosages, allergies, fears, the vet's number, the emotional moving averages. They exist only in Hermes memory on the owner's machine. The Worker can forward a photo URL and receive a verdict, but it never sees the accumulated record those verdicts build. Ask the Worker "what is wrong with my dog this week" and it cannot answer; the daily review that could, runs at home and pushes to your WeChat directly.

The PIN, ever, in readable form. Care verification is built so that even the service storing the credentials cannot read them back. A created session stores pin_hash — SHA-256(pin + server secret) — not the PIN. Verification compares hashes with a timing-safe equality check (timingSafeEqual over a byte-wise XOR), throttles to five attempts per minute, and locks the code for an hour after ten failures. A database leak yields hashes with a server secret, not a stack of four-digit PINs.

The caregiver's identity. The most important privacy property of the care system is that the sitter is not a record. The database stores uploader: 'caregiver' — a role label. No name, no phone, no email, no device ID, no behavioral profile. The care token that lets a sitter upload is a signed capability: payload {care_code}|{care_session_id}|{exp}, HMAC-SHA256 with CARE_CODE_SECRET, formatted {care_code}.{exp}.{sig}, validated on three conditions — signature, expiry, and an active session in KV. A sitter who types a code and PIN into the H5 page becomes, in the system's eyes, a token holder. When the session ends, the token dies, and there is nothing left to delete because nothing personal was ever stored.

The private conversations of care. The caregiver note attached to a photo ("she did not eat breakfast", "he seems tired today") travels in the webhook payload and lands in D1 as caregiver_note — a fact about the dog, attached to a role, not a person. The care handbook, meanwhile, follows the iron rule from the care-handbook skill: dosage, medication, vet and emergency numbers are copied verbatim from the profile and never exposed beyond what the sitter needs — the sitter's handbook deliberately omits the owner's private backup contact number, stating only "in case of emergency, contact the vet immediately."

Notice the pattern. Everything the Worker holds is either (a) needed to move bytes or enforce limits, (b) short-lived, or (c) both. Everything with lasting meaning — the biography, the trends, the voice, the key — stays home. That asymmetry is not an accident of implementation; it is the design. The photo check-in deep dive shows the full path a photo takes, and you can trace every byte of it against the two lists above: the cloud holds the copy needed to deliver the analysis, and the meaning accumulates where you live.

Care Codes and the Missing Social Graph

Sharing is where most pet apps betray their privacy promises, because sharing usually means registration. PupPal's care system was built on the opposite assumption: possession of a code is the only credential that should ever matter.

When an owner starts a care session, the Worker generates a nine-digit care code, modeled explicitly on the RustDesk connection code and grouped 3-3-3 so it can be read aloud over the phone — "nine-oh-eight, one-seven-four, five-five-three" — plus a four-digit PIN. The PIN is never stored; only its salted hash is, and the verification path is hardened with the rate limits and lockout described above. The sitter does not install the app and does not create an account. They open the H5 care page (served as a static HTML shell at the /c/ route), type the code and PIN, and get the care handbook plus a signed upload token valid for at most seven days and capped by the session window. That is the entire onboarding: two numbers, one page, zero identity.

The care session record ties a code to a dog and a time window — dog_id, start_time, end_time, status, transfer_mode, service_type — not to a human. There is no friend model, no follower list, no "people you know" discovery, no social graph of any kind in the care path. The worker cannot answer "who is caring for this dog"; it can only answer "does this token belong to an active session for this dog." That is a feature, not a limitation: a graph of human relationships is exactly the kind of data a pet app should never need to own, because it multiplies the privacy surface for every human in it.

Even the newer sharing surfaces in the codebase stay on the capability side of the line. The family circle feature — cross-user shared check-ins with likes and comments — is built on explicit invite codes (eight uppercase characters derived from a group UUID) and membership rows checked on every request; only records explicitly flagged is_shared are ever synchronized, and the real-time path is WebRTC signal rooms, not a content silo. Invitation, not discovery. The shape of every sharing feature is the same: the owner hands out a small secret, the recipient uses it, and the system records the minimum needed to keep the session honest. The care codes and PIN post walks the full security model — hashing, throttling, lockout, token expiry — and it is worth reading as the privacy contract of the sharing feature: the design protects the sitter's anonymity and the owner's dog at the same time, because neither side ever hands the platform anything it can resell.

There is a quiet elegance to this for pet data privacy. In a registration-based app, the sitter becomes a permanent data point — a profile, a device, a habit history — long after the weekend of dog-sitting is over. With care codes, the sitter is a phantom: they appear at the door, perform the care, and vanish from the record when the session expires, leaving only photos attributed to a role and a dog. The person who helped you is not a product the platform now owns.

Privacy as a Feature: Why This Design Wins

Privacy by design is usually discussed as a constraint — all the clever things you cannot do. PupPal treats it as an engineering property with concrete payoffs, and the payoffs are worth making explicit, because they are the reasons the design is not just ethical but good.

The first payoff is graceful degradation. Because the AI lives at home, the product does not depend on a vendor's API being up, a cloud region being reachable, or a data pipeline being healthy. When Hermes is asleep or offline, the relay's tryHermesWebhook wrapper swallows the failure and the core care loop still works: a care code can be created, verified, and ended without any AI involvement, and the sitter still gets the handbook snapshot that was captured at session creation. Privacy and resilience are the same property here: you cannot be cut off from a service you host. The foster care monitoring post and the self-hosted agent skills post both show how much of the product runs entirely on that home side of the line.

The second payoff is data minimization as a default. Every piece of data the Worker holds has a TTL, a cleanup path, or both. Care sessions expire their KV keys in thirty days; rate-limit counters self-expire; staged photos are deleted on pull; care-session R2 binaries are swept seven days after the session ends; upload tokens die with the session. The codebase even records, per photo, which delivery route it took — direct, staged, or cloud — so a photo that went phone-to-phone over the WebRTC DataChannel (the app's Rust core uses webrtc-rs for exactly this) is a photo the cloud never stored at all. Minimization is a property you can audit in the code, not a promise in a policy.

The third payoff is trust that scales to real humans. Every human in the system — the owner, the sitter, the grandmother with the care code, the foster family with the sharper thresholds — is protected by the same structural choice: no account, no identity, no graph. When you hand your dog to someone with a nine-digit code, you are not handing them a data breach risk, and they are not handing you a lifetime of platform membership. The dog care handbook story shows what that feels like from the caregiver's side: a person who has never met your dog can still care for it correctly, because the knowledge travels as a document, not as an account.

None of this requires you to be paranoid, and none of it requires you to be an engineer. It requires the product to be built the way PupPal is built: the brain at home, the relay thin, the sharing anonymous. The Cloudflare Worker is a real service with a real attack surface — and the design's answer to that fact is not a longer privacy policy, it is a smaller attack surface. There is less to steal because there is less to hold. There is less to subpoena because there is less to know. There is less to leak because the biography of your dog never left your house.

So try the experiment that this whole architecture was built for. Tomorrow morning, before work, point PupPal at your dog's breakfast bowl and take the photo. Then ask yourself where that photo's meaning was created. It was not created in a data center; it was created on a machine near your kitchen, by an agent you host, using a key you own, writing to a memory that describes one animal and nothing else. That is what pet data privacy looks like when it is engineered instead of promised. One photo, one check-in, one day at a time — and the story of your dog stays exactly where it belongs.