# I Wanted to Keep Track of My App Reviews Without Expensive Tooling — So I Built My Own



> **TL;DR:** I maintain a couple of apps on both stores. I kept losing track of what users were saying, and every tool I found either cost too much or did too many things I didn't need. So I wrote a CLI that pulls my reviews, classifies them with an LLM, and tells me what broke. It runs locally, costs me ~$0.50/month in API calls, and the code is on [GitHub](https://github.com/Mr-Ashish/AppPulse) if any of this is useful to you.

---

## What Was Bugging Me

I maintain a few apps on Google Play and the App Store. Nothing huge — just apps I've built over the years that have real users. Every few days I'd remember to open Play Console, scroll through reviews, and try to mentally sort them — "ok that's a bug, that's a feature request, that person is just frustrated, that one is a crash I already fixed three releases ago."

I kept missing stuff. A user would report something important, and I'd only see it two weeks later buried under a dozen "great app 5 stars" reviews. I knew what I actually needed:

- **Fetch my reviews** from both stores without me having to open two dashboards
- **Fetch ratings** so I'd notice a drop before it snowballs
- **Figure out what's a bug and what's a feature request** — because I don't have time to read every review
- **Categorize them properly** — not just sentiment, but actual categories: bug, crash, feature request, performance issue
- **Correlate crashes if possible** — when someone writes "app keeps closing," I want to know if there's a matching crash in Sentry
- **Just alert me** — a morning summary in Slack or my terminal. I don't need a dashboard. I need a nudge.

## I Looked Around for a While

Before writing any code, I spent a fair amount of time looking for something that already existed. Surely someone had solved this.

I tried **AppFollow**, **Appbot**, **AppTweak**, and a few others. They range from $50 to $200+ a month. Some of them are genuinely good products — if you're a team with a PM who needs dashboards, competitor tracking, ASO tools, and review assignment workflows. But I'm one developer trying to keep tabs on my own apps. I don't need competitor intelligence. I don't need ASO. I just need to know what broke and what people are asking for.

The classification in most of these tools was also frustrating. They mostly use keyword matching, which means a review like "this used to crash all the time but the update fixed it" gets flagged as a crash report. That's noise, not signal.

I also tried just being more disciplined about checking Play Console and App Store Connect manually. That lasted about two weeks before I was back to checking once every few days and missing things.

What I really wanted didn't seem to exist: something lightweight, something I could run locally, something that actually understood what a review was saying — not just pattern-matching on keywords. And ideally something I wasn't paying a monthly subscription for.

So I figured I'd just build it.

## What I Ended Up Building

I called it [AppPulse](https://github.com/Mr-Ashish/AppPulse). It's a Python CLI — no web app, no Docker, no hosted service. Just `pip install` and run it on your machine.

Here's what happens when I run `apppulse run` (or when cron triggers it at 7 AM):

1. **Pulls new reviews** from Google Play and/or the App Store via their official APIs
2. **Classifies each review** using an LLM — bug, feature request, crash report, performance complaint, praise, or general frustration
3. **Pulls crash data** from Sentry so I can see if user complaints match real crashes
4. **Stores everything** in a local SQLite database
5. **Generates a digest** — top bugs, top feature requests, crash health
6. **Sends it** to my terminal, Slack, or email

```
pip install -e .
apppulse setup    # guided wizard, ~5 minutes
apppulse run      # pull → classify → report
```

No servers, no cloud accounts. Everything stays in a SQLite file on my machine at `~/.apppulse/data.db`.

## How It Fits Together

```
           apppulse run (or daily at 7 AM)
                      │
         ┌────────────┼────────────┐
         ▼                  ▼                  ▼
       Google Play       App Store             Sentry
      Publisher API     Connect API           REST API
         │                   │                   │
         └────────────┼────────────┘
                      ▼
              ┌──────────────┐
              │  Raw Reviews │
              │ + Crash Data │
              └──────┬───────┘
                     ▼
              ┌──────────────┐
              │ LLM Classify │  ← GPT-4o-mini
              │ (batch of 10)│
              └──────┬───────┘
                     ▼
              ┌──────────────┐
              │  SQLite DB   │  ← ~/.apppulse/data.db
              └──────┬───────┘
                     ▼
           ┌─────────┼─────────┐
           ▼             ▼             ▼
        Terminal         Slack         Email
```

I kept each step as a separate module, mostly so I could swap things out as I went. If you want to add a different crash source or use a different LLM, you'd change one file.

## The Part That Made It Worth Building

I could have just written a script to dump reviews into a CSV. But the whole point was that I didn't want to read every review myself. I wanted the tool to tell me: "these 4 are bugs, these 2 are feature requests, and the rest are just people saying thanks."

Keyword matching doesn't cut it. A review saying "the app never crashes anymore" is praise, not a crash report. So AppPulse sends reviews to an LLM in batches of 10 and gets back structured results:

```python
# From src/apppulse/analyze/classifier.py
# Each review gets classified into:
{
    "category": "bug",           # bug, feature_request, crash, performance, praise, complaint
    "severity": "major",         # critical, major, minor, none
    "sentiment": "negative",     # positive, negative, neutral, mixed
    "summary": "App freezes when uploading photos larger than 10MB",
    "keywords": ["freeze", "upload", "photos"],
    "functional_area": "media"   # which part of your app is affected
}
```

The batch approach matters for cost. One API call handles 10 reviews instead of 10 separate calls. With GPT-4o-mini, my apps generate maybe 1,000-2,000 reviews a month, and the LLM bill comes to around **$0.50**. That's the entire monthly cost of running this.

If you're not comfortable sending review text to OpenAI or Anthropic, there's an **Ollama** option — run Llama 3.2 locally and classification is completely free.

## The Setup (Not as Bad as It Sounds)

You need Python 3.10+, API credentials for the stores (a Google Cloud Service Account for Play, a `.p8` key for App Store Connect), and an LLM provider (OpenAI API key, or just Ollama running locally).

### Install

```bash
git clone https://github.com/Mr-Ashish/AppPulse.git
cd AppPulse
pip install -e .
```

### Run the Setup Wizard

```bash
apppulse setup
```

The wizard walks through 4 steps — adding apps, choosing an LLM, setting up notifications, configuring the schedule. I put inline instructions in each step because I kept forgetting which Play Console page to visit for the service account. Now it just tells me.

After the wizard, you set a couple of environment variables (API keys — these are never stored in the config file):

```bash
# ~/.zshrc or ~/.bashrc
export OPENAI_API_KEY="sk-..."
export SENTRY_AUTH_TOKEN="sntrys_..."           # optional
export APPPULSE_SLACK_WEBHOOK="https://hooks.slack.com/..."  # optional
```

Then verify and run:

```bash
apppulse test   # checks all credentials exist
apppulse run    # pulls reviews, classifies, prints digest
```

The first run grabs the last ~7 days of Google Play reviews (that's a hard API limit — more on that below) and all available App Store reviews.

## The 7-Day Problem (This One Annoyed Me)

This caught me off guard. The **Google Play Publisher API only returns the last 7 days of reviews**. There's no parameter to go further back. It's a hard limit baked into the API.

For an app that's been live for two years, that means thousands of reviews you can never pull through the API. I wanted the full history — especially to see patterns over time and to have a proper baseline for trend detection.

After digging around, I found that Google has an official **GCS bulk export** option. Play Console can export your entire review history as monthly CSV files to a private Google Cloud Storage bucket. These go back to the day the app was first published.

So I added a `backfill` command that:

1. Connects to your GCS bucket using the same service account
2. Lists all available monthly CSV files
3. Downloads and parses them (handling Google's UTF-16 encoding)
4. Deduplicates against reviews already in your database
5. Optionally classifies them with your LLM

```bash
# Install the GCS dependency
pip install -e ".[gcs]"

# Backfill everything
apppulse backfill --app "MyApp Android"

# Backfill from a specific date
apppulse backfill --app "MyApp Android" --since 2024-01-01

# Preview without importing
apppulse backfill --app "MyApp Android" --dry-run
```

To enable it, add the GCS bucket ID to your config (`~/.apppulse/config.yaml`):

```yaml
apps:
  - name: "MyApp Android"
    platform: "google_play"
    package_name: "com.example.myapp"
    gcs_bucket: "pubsite_prod_rev_12345678901234567890"
    credentials:
      service_account_json: "/path/to/key.json"
```

**Watch out:** The GCS bucket permission requires **account-level** access in Play Console, not just app-level. And it can take up to **24 hours** to propagate after you grant it. If you get a `storage.objects.list` permission error, wait a day before debugging further.

## What I Actually Use Day to Day

It's all terminal-based. Here's what I have:

| Command | What it does |
|---------|-------------|
| `apppulse setup` | Interactive wizard — credentials, LLM, notifications, schedule |
| `apppulse run` | One-shot: pull + classify + digest + notify |
| `apppulse start` | Start the daily scheduler (runs at your configured time) |
| `apppulse backfill --app "X"` | Import full review history from GCS |
| `apppulse reviews --category bug` | Browse reviews filtered by category |
| `apppulse reviews --rating 1 --days 30` | Find all 1-star reviews in the last month |
| `apppulse crashes` | Show top crashes from Sentry |
| `apppulse reply 42 "Thanks!"` | Reply to a review by its DB ID |
| `apppulse export --format csv` | Export reviews to CSV or JSON |
| `apppulse test` | Verify all credentials are working |

## What It Doesn't Do

I should be honest about this — it's a CLI tool I built for my own use. There's no web dashboard (yet). There's no team collaboration, no review assignment workflow, no competitor tracking. If you need to share review reports with a PM who doesn't use a terminal, this isn't the tool for that right now.

If you have a team that needs shared dashboards, role-based access, or competitor intelligence, the paid platforms are genuinely good at that. I just didn't need any of it.

What I do get is: my data stays on my machine in a SQLite file I can query directly, the LLM-based classification actually understands context (not just keywords), and I can hook in my Sentry crash data — something none of the tools I looked at could do. And the whole thing costs me about $0.50/month in OpenAI API calls. If even that feels like too much, Ollama runs locally for free.

## What I'm Working on Next

This is still a work in progress. Here's what I want to add for my own use — and if any of this is useful to you, PRs are welcome.

**Things I'm actively building:**
- **Review-crash correlation** — automatically match 1-star reviews with Sentry crash reports by keyword overlap, device info, timing, and version
- **Trend detection** — notice when bug reports spike after a release, or when ratings start dropping
- **Background daemon mode** — so the scheduler runs without blocking my terminal

**Things I'd like to get to eventually:**
- **A simple web dashboard** — probably Streamlit. I don't need something fancy, just a visual overview of what's happening across my apps
- **Review clustering** — so when 20 people report the same login bug, it shows up as one item, not 20
- **Auto-drafted replies** — generate a reply based on the category, let me approve before sending
- **GitHub issue creation** — when a cluster of reviews hits a threshold, auto-create an issue with excerpts and device breakdown

If you want to pick up any of these, the [issues page](https://github.com/Mr-Ashish/AppPulse/issues) is the best place to start.

## If You Want to Try It

The whole thing is on GitHub: **[github.com/Mr-Ashish/AppPulse](https://github.com/Mr-Ashish/AppPulse)**

```bash
git clone https://github.com/Mr-Ashish/AppPulse.git
cd AppPulse
pip install -e .
apppulse setup
```

It's MIT licensed — fork it, extend it, gut it and use the parts you need. If you find bugs or have ideas, [open an issue](https://github.com/Mr-Ashish/AppPulse/issues). I built this to solve my own problem, but if it saves you some time or money too, that's great.

---

*[Ashish Mishra](https://github.com/Mr-Ashish) — I build apps and occasionally build tools to avoid reading app reviews manually.*