Add game data seeding from PokeAPI with level ranges
Seed the database with Pokemon game data for 5 games (FireRed, LeafGreen, Emerald, HeartGold, SoulSilver) using pokebase. Includes Alembic migrations for route unique constraints and encounter level ranges, a two-phase seed system (offline fetch to JSON, then idempotent upserts), and Dockerfile updates for the seed runner. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
11
.beans/nuzlocke-tracker-igl3--name-generation.md
Normal file
11
.beans/nuzlocke-tracker-igl3--name-generation.md
Normal file
@@ -0,0 +1,11 @@
|
|||||||
|
---
|
||||||
|
# nuzlocke-tracker-igl3
|
||||||
|
title: Name Generation
|
||||||
|
status: draft
|
||||||
|
type: feature
|
||||||
|
priority: normal
|
||||||
|
created_at: 2026-02-05T13:45:15Z
|
||||||
|
updated_at: 2026-02-05T13:46:30Z
|
||||||
|
---
|
||||||
|
|
||||||
|
For nuzlockes I want to implement name generation. The user should be able to provide a naming scheme or a list of nick names that can then be selected when a new encounter is registered.
|
||||||
@@ -0,0 +1,26 @@
|
|||||||
|
---
|
||||||
|
# nuzlocke-tracker-j28y
|
||||||
|
title: Curate route ordering to match game progression
|
||||||
|
status: todo
|
||||||
|
type: task
|
||||||
|
created_at: 2026-02-05T13:37:39Z
|
||||||
|
updated_at: 2026-02-05T13:37:39Z
|
||||||
|
parent: nuzlocke-tracker-f5ob
|
||||||
|
---
|
||||||
|
|
||||||
|
Routes are currently in alphabetical order from PokeAPI. Update the order field in each game's JSON seed file to reflect actual game progression (e.g., Pallet Town → Route 1 → Viridian City → Route 2 → ...).
|
||||||
|
|
||||||
|
## Details
|
||||||
|
- 646 routes total across 5 games
|
||||||
|
- FireRed and LeafGreen share the same route progression (Kanto)
|
||||||
|
- HeartGold and SoulSilver share the same route progression (Johto + Kanto)
|
||||||
|
- Emerald has its own progression (Hoenn)
|
||||||
|
- So effectively 3 unique orderings to define
|
||||||
|
- After updating JSON files, re-run the seed: `podman compose exec -e PYTHONUNBUFFERED=1 -w /app/src api python -m app.seeds`
|
||||||
|
|
||||||
|
## Files
|
||||||
|
- `backend/src/app/seeds/data/firered.json`
|
||||||
|
- `backend/src/app/seeds/data/leafgreen.json`
|
||||||
|
- `backend/src/app/seeds/data/emerald.json`
|
||||||
|
- `backend/src/app/seeds/data/heartgold.json`
|
||||||
|
- `backend/src/app/seeds/data/soulsilver.json`
|
||||||
@@ -1,31 +1,34 @@
|
|||||||
---
|
---
|
||||||
# nuzlocke-tracker-k5lm
|
# nuzlocke-tracker-k5lm
|
||||||
title: Initial Game Data Seeding
|
title: Initial Game Data Seeding
|
||||||
status: todo
|
status: completed
|
||||||
type: task
|
type: task
|
||||||
priority: normal
|
priority: normal
|
||||||
created_at: 2026-02-04T15:44:12Z
|
created_at: 2026-02-04T15:44:12Z
|
||||||
updated_at: 2026-02-04T15:47:29Z
|
updated_at: 2026-02-05T13:37:50Z
|
||||||
parent: nuzlocke-tracker-f5ob
|
parent: nuzlocke-tracker-f5ob
|
||||||
---
|
---
|
||||||
|
|
||||||
Create seed data for the database with initial games, routes, and Pokémon.
|
Create seed data for the database with initial games, routes, and Pokémon.
|
||||||
|
|
||||||
## Checklist
|
## Checklist
|
||||||
- [ ] Research and compile data for MVP games:
|
- [x] Research and compile data for MVP games:
|
||||||
- [ ] Pokémon FireRed/LeafGreen (Gen 3 Kanto remakes - popular for Nuzlockes)
|
- [x] Pokémon FireRed/LeafGreen (Gen 3 Kanto remakes - popular for Nuzlockes)
|
||||||
- [ ] Pokémon Emerald (Gen 3 Hoenn)
|
- [x] Pokémon Emerald (Gen 3 Hoenn)
|
||||||
- [ ] Pokémon HeartGold/SoulSilver (Gen 4 Johto remakes)
|
- [x] Pokémon HeartGold/SoulSilver (Gen 4 Johto remakes)
|
||||||
- [ ] For each game, gather:
|
- [x] For each game, gather:
|
||||||
- [ ] All routes/areas in progression order
|
- [x] All routes/areas in progression order
|
||||||
- [ ] Available wild Pokémon per route
|
- [x] Available wild Pokémon per route
|
||||||
- [ ] Encounter methods (grass, surf, fish, etc.)
|
- [x] Encounter methods (grass, surf, fish, etc.)
|
||||||
- [ ] Create seed scripts/migrations to populate database
|
- [x] Create seed scripts/migrations to populate database
|
||||||
- [ ] Include Pokémon base data (national dex, names, types, sprite URLs)
|
- [x] Include Pokémon base data (national dex, names, types, sprite URLs)
|
||||||
- [ ] Document data sources for attribution
|
- [x] Document data sources for attribution
|
||||||
|
- [x] Curate route ordering to match game progression — split to nuzlocke-tracker-j28y
|
||||||
|
|
||||||
## Notes
|
## Notes
|
||||||
- Use PokeAPI or Bulbapedia as data sources
|
|
||||||
- Admin panel allows adding more games later
|
- Admin panel allows adding more games later
|
||||||
- Focus on accuracy for the 3 MVP games
|
- Focus on accuracy for the 3 MVP games
|
||||||
- Sprite URLs can point to existing sprite repositories
|
|
||||||
|
## Data Sources
|
||||||
|
- Game data (routes, encounters, Pokemon): [PokeAPI](https://pokeapi.co/) via [pokebase](https://github.com/PokeAPI/pokebase) Python wrapper
|
||||||
|
- Sprites: [PokeAPI/sprites](https://github.com/PokeAPI/sprites) on GitHub
|
||||||
@@ -9,7 +9,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
|
|||||||
&& rm -rf /var/lib/apt/lists/*
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
# Install Python dependencies
|
# Install Python dependencies
|
||||||
COPY pyproject.toml README.md ./
|
COPY pyproject.toml README.md alembic.ini ./
|
||||||
COPY src/ ./src/
|
COPY src/ ./src/
|
||||||
|
|
||||||
RUN pip install --no-cache-dir -e .
|
RUN pip install --no-cache-dir -e .
|
||||||
|
|||||||
@@ -21,6 +21,7 @@ dev = [
|
|||||||
"pytest>=8.0.0",
|
"pytest>=8.0.0",
|
||||||
"pytest-asyncio>=0.25.0",
|
"pytest-asyncio>=0.25.0",
|
||||||
"httpx>=0.28.0",
|
"httpx>=0.28.0",
|
||||||
|
"pokebase>=1.4.0",
|
||||||
]
|
]
|
||||||
|
|
||||||
[build-system]
|
[build-system]
|
||||||
|
|||||||
@@ -0,0 +1,32 @@
|
|||||||
|
"""add unique constraint routes game name
|
||||||
|
|
||||||
|
Revision ID: 694df688fb02
|
||||||
|
Revises: 03e5f186a9d5
|
||||||
|
Create Date: 2026-02-05 13:01:30.631978
|
||||||
|
|
||||||
|
"""
|
||||||
|
from typing import Sequence, Union
|
||||||
|
|
||||||
|
from alembic import op
|
||||||
|
import sqlalchemy as sa
|
||||||
|
|
||||||
|
|
||||||
|
# revision identifiers, used by Alembic.
|
||||||
|
revision: str = '694df688fb02'
|
||||||
|
down_revision: Union[str, Sequence[str], None] = '03e5f186a9d5'
|
||||||
|
branch_labels: Union[str, Sequence[str], None] = None
|
||||||
|
depends_on: Union[str, Sequence[str], None] = None
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
"""Upgrade schema."""
|
||||||
|
# ### commands auto generated by Alembic - please adjust! ###
|
||||||
|
op.create_unique_constraint('uq_routes_game_name', 'routes', ['game_id', 'name'])
|
||||||
|
# ### end Alembic commands ###
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
"""Downgrade schema."""
|
||||||
|
# ### commands auto generated by Alembic - please adjust! ###
|
||||||
|
op.drop_constraint('uq_routes_game_name', 'routes', type_='unique')
|
||||||
|
# ### end Alembic commands ###
|
||||||
@@ -0,0 +1,34 @@
|
|||||||
|
"""add level range to route encounters
|
||||||
|
|
||||||
|
Revision ID: 9afcbafe9888
|
||||||
|
Revises: 694df688fb02
|
||||||
|
Create Date: 2026-02-05 13:32:35.559499
|
||||||
|
|
||||||
|
"""
|
||||||
|
from typing import Sequence, Union
|
||||||
|
|
||||||
|
from alembic import op
|
||||||
|
import sqlalchemy as sa
|
||||||
|
|
||||||
|
|
||||||
|
# revision identifiers, used by Alembic.
|
||||||
|
revision: str = '9afcbafe9888'
|
||||||
|
down_revision: Union[str, Sequence[str], None] = '694df688fb02'
|
||||||
|
branch_labels: Union[str, Sequence[str], None] = None
|
||||||
|
depends_on: Union[str, Sequence[str], None] = None
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
"""Upgrade schema."""
|
||||||
|
op.add_column('route_encounters', sa.Column('min_level', sa.SmallInteger(), nullable=False, server_default='0'))
|
||||||
|
op.add_column('route_encounters', sa.Column('max_level', sa.SmallInteger(), nullable=False, server_default='0'))
|
||||||
|
op.alter_column('route_encounters', 'min_level', server_default=None)
|
||||||
|
op.alter_column('route_encounters', 'max_level', server_default=None)
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
"""Downgrade schema."""
|
||||||
|
# ### commands auto generated by Alembic - please adjust! ###
|
||||||
|
op.drop_column('route_encounters', 'max_level')
|
||||||
|
op.drop_column('route_encounters', 'min_level')
|
||||||
|
# ### end Alembic commands ###
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
from sqlalchemy import ForeignKey, SmallInteger, String
|
from sqlalchemy import ForeignKey, SmallInteger, String, UniqueConstraint
|
||||||
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||||
|
|
||||||
from app.core.database import Base
|
from app.core.database import Base
|
||||||
@@ -6,6 +6,9 @@ from app.core.database import Base
|
|||||||
|
|
||||||
class Route(Base):
|
class Route(Base):
|
||||||
__tablename__ = "routes"
|
__tablename__ = "routes"
|
||||||
|
__table_args__ = (
|
||||||
|
UniqueConstraint("game_id", "name", name="uq_routes_game_name"),
|
||||||
|
)
|
||||||
|
|
||||||
id: Mapped[int] = mapped_column(primary_key=True)
|
id: Mapped[int] = mapped_column(primary_key=True)
|
||||||
name: Mapped[str] = mapped_column(String(100))
|
name: Mapped[str] = mapped_column(String(100))
|
||||||
|
|||||||
@@ -17,6 +17,8 @@ class RouteEncounter(Base):
|
|||||||
pokemon_id: Mapped[int] = mapped_column(ForeignKey("pokemon.id"), index=True)
|
pokemon_id: Mapped[int] = mapped_column(ForeignKey("pokemon.id"), index=True)
|
||||||
encounter_method: Mapped[str] = mapped_column(String(30))
|
encounter_method: Mapped[str] = mapped_column(String(30))
|
||||||
encounter_rate: Mapped[int] = mapped_column(SmallInteger)
|
encounter_rate: Mapped[int] = mapped_column(SmallInteger)
|
||||||
|
min_level: Mapped[int] = mapped_column(SmallInteger)
|
||||||
|
max_level: Mapped[int] = mapped_column(SmallInteger)
|
||||||
|
|
||||||
route: Mapped["Route"] = relationship(back_populates="route_encounters")
|
route: Mapped["Route"] = relationship(back_populates="route_encounters")
|
||||||
pokemon: Mapped["Pokemon"] = relationship(back_populates="route_encounters")
|
pokemon: Mapped["Pokemon"] = relationship(back_populates="route_encounters")
|
||||||
|
|||||||
0
backend/src/app/seeds/__init__.py
Normal file
0
backend/src/app/seeds/__init__.py
Normal file
21
backend/src/app/seeds/__main__.py
Normal file
21
backend/src/app/seeds/__main__.py
Normal file
@@ -0,0 +1,21 @@
|
|||||||
|
"""Entry point for running seeds.
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
python -m app.seeds # Run seed
|
||||||
|
python -m app.seeds --verify # Run seed + verification
|
||||||
|
"""
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import sys
|
||||||
|
|
||||||
|
from app.seeds.run import seed, verify
|
||||||
|
|
||||||
|
|
||||||
|
async def main():
|
||||||
|
await seed()
|
||||||
|
if "--verify" in sys.argv:
|
||||||
|
await verify()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
asyncio.run(main())
|
||||||
6342
backend/src/app/seeds/data/emerald.json
Normal file
6342
backend/src/app/seeds/data/emerald.json
Normal file
File diff suppressed because it is too large
Load Diff
8550
backend/src/app/seeds/data/firered.json
Normal file
8550
backend/src/app/seeds/data/firered.json
Normal file
File diff suppressed because it is too large
Load Diff
37
backend/src/app/seeds/data/games.json
Normal file
37
backend/src/app/seeds/data/games.json
Normal file
@@ -0,0 +1,37 @@
|
|||||||
|
[
|
||||||
|
{
|
||||||
|
"name": "Pokemon FireRed",
|
||||||
|
"slug": "firered",
|
||||||
|
"generation": 3,
|
||||||
|
"region": "kanto",
|
||||||
|
"release_year": 2004
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "Pokemon LeafGreen",
|
||||||
|
"slug": "leafgreen",
|
||||||
|
"generation": 3,
|
||||||
|
"region": "kanto",
|
||||||
|
"release_year": 2004
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "Pokemon Emerald",
|
||||||
|
"slug": "emerald",
|
||||||
|
"generation": 3,
|
||||||
|
"region": "hoenn",
|
||||||
|
"release_year": 2005
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "Pokemon HeartGold",
|
||||||
|
"slug": "heartgold",
|
||||||
|
"generation": 4,
|
||||||
|
"region": "johto",
|
||||||
|
"release_year": 2010
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "Pokemon SoulSilver",
|
||||||
|
"slug": "soulsilver",
|
||||||
|
"generation": 4,
|
||||||
|
"region": "johto",
|
||||||
|
"release_year": 2010
|
||||||
|
}
|
||||||
|
]
|
||||||
15290
backend/src/app/seeds/data/heartgold.json
Normal file
15290
backend/src/app/seeds/data/heartgold.json
Normal file
File diff suppressed because it is too large
Load Diff
8566
backend/src/app/seeds/data/leafgreen.json
Normal file
8566
backend/src/app/seeds/data/leafgreen.json
Normal file
File diff suppressed because it is too large
Load Diff
2036
backend/src/app/seeds/data/pokemon.json
Normal file
2036
backend/src/app/seeds/data/pokemon.json
Normal file
File diff suppressed because it is too large
Load Diff
15330
backend/src/app/seeds/data/soulsilver.json
Normal file
15330
backend/src/app/seeds/data/soulsilver.json
Normal file
File diff suppressed because it is too large
Load Diff
329
backend/src/app/seeds/fetch_pokeapi.py
Normal file
329
backend/src/app/seeds/fetch_pokeapi.py
Normal file
@@ -0,0 +1,329 @@
|
|||||||
|
"""Fetch game data from PokeAPI and write static JSON seed files.
|
||||||
|
|
||||||
|
Uses pokebase which provides built-in file caching — first run fetches
|
||||||
|
from the API, subsequent runs are instant from disk cache.
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
# Against public PokeAPI (cached after first run):
|
||||||
|
podman compose exec -w /app/src api python -m app.seeds.fetch_pokeapi
|
||||||
|
|
||||||
|
# Against local PokeAPI (no rate limits):
|
||||||
|
podman compose exec -w /app/src api python -m app.seeds.fetch_pokeapi --base-url http://pokeapi-app:8000
|
||||||
|
"""
|
||||||
|
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pokebase as pb
|
||||||
|
import pokebase.common as pb_common
|
||||||
|
|
||||||
|
DATA_DIR = Path(__file__).parent / "data"
|
||||||
|
|
||||||
|
# Game definitions
|
||||||
|
VERSION_GROUPS = {
|
||||||
|
"firered-leafgreen": {
|
||||||
|
"versions": ["firered", "leafgreen"],
|
||||||
|
"generation": 3,
|
||||||
|
"region": "kanto",
|
||||||
|
"region_id": 1,
|
||||||
|
"games": {
|
||||||
|
"firered": {
|
||||||
|
"name": "Pokemon FireRed",
|
||||||
|
"slug": "firered",
|
||||||
|
"release_year": 2004,
|
||||||
|
},
|
||||||
|
"leafgreen": {
|
||||||
|
"name": "Pokemon LeafGreen",
|
||||||
|
"slug": "leafgreen",
|
||||||
|
"release_year": 2004,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
"emerald": {
|
||||||
|
"versions": ["emerald"],
|
||||||
|
"generation": 3,
|
||||||
|
"region": "hoenn",
|
||||||
|
"region_id": 3,
|
||||||
|
"games": {
|
||||||
|
"emerald": {
|
||||||
|
"name": "Pokemon Emerald",
|
||||||
|
"slug": "emerald",
|
||||||
|
"release_year": 2005,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
"heartgold-soulsilver": {
|
||||||
|
"versions": ["heartgold", "soulsilver"],
|
||||||
|
"generation": 4,
|
||||||
|
"region": "johto",
|
||||||
|
"region_id": 2,
|
||||||
|
"games": {
|
||||||
|
"heartgold": {
|
||||||
|
"name": "Pokemon HeartGold",
|
||||||
|
"slug": "heartgold",
|
||||||
|
"release_year": 2010,
|
||||||
|
},
|
||||||
|
"soulsilver": {
|
||||||
|
"name": "Pokemon SoulSilver",
|
||||||
|
"slug": "soulsilver",
|
||||||
|
"release_year": 2010,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
# Encounter methods to include (excludes gift, legendary-only, etc.)
|
||||||
|
INCLUDED_METHODS = {
|
||||||
|
"walk",
|
||||||
|
"surf",
|
||||||
|
"old-rod",
|
||||||
|
"good-rod",
|
||||||
|
"super-rod",
|
||||||
|
"rock-smash",
|
||||||
|
"headbutt",
|
||||||
|
}
|
||||||
|
|
||||||
|
# Collect all pokemon dex numbers across games
|
||||||
|
all_pokemon_dex: set[int] = set()
|
||||||
|
|
||||||
|
|
||||||
|
def clean_location_name(name: str) -> str:
|
||||||
|
"""Convert PokeAPI location slug to a clean display name.
|
||||||
|
|
||||||
|
e.g. 'kanto-route-1' -> 'Route 1'
|
||||||
|
'pallet-town' -> 'Pallet Town'
|
||||||
|
"""
|
||||||
|
for prefix in [
|
||||||
|
"kanto-", "johto-", "hoenn-", "sinnoh-",
|
||||||
|
"unova-", "kalos-", "alola-", "galar-",
|
||||||
|
]:
|
||||||
|
if name.startswith(prefix):
|
||||||
|
name = name[len(prefix):]
|
||||||
|
break
|
||||||
|
|
||||||
|
name = name.replace("-", " ").title()
|
||||||
|
name = re.sub(r"Route (\d+)", r"Route \1", name)
|
||||||
|
return name
|
||||||
|
|
||||||
|
|
||||||
|
def clean_area_name(area_name: str, location_name: str) -> str | None:
|
||||||
|
"""Extract meaningful area suffix, or None if it's the default area."""
|
||||||
|
if area_name.startswith(location_name):
|
||||||
|
suffix = area_name[len(location_name):].strip("-").strip()
|
||||||
|
if not suffix or suffix == "area":
|
||||||
|
return None
|
||||||
|
return suffix.replace("-", " ").title()
|
||||||
|
return area_name.replace("-", " ").title()
|
||||||
|
|
||||||
|
|
||||||
|
def get_encounters_for_area(area_id: int, version_name: str) -> list[dict]:
|
||||||
|
"""Get encounter data for a location area, filtered by version."""
|
||||||
|
area = pb.location_area(area_id)
|
||||||
|
encounters = []
|
||||||
|
|
||||||
|
for pe in area.pokemon_encounters:
|
||||||
|
pokemon_url = pe.pokemon.url
|
||||||
|
dex_num = int(pokemon_url.rstrip("/").split("/")[-1])
|
||||||
|
pokemon_name = pe.pokemon.name
|
||||||
|
|
||||||
|
for vd in pe.version_details:
|
||||||
|
if vd.version.name != version_name:
|
||||||
|
continue
|
||||||
|
|
||||||
|
for enc in vd.encounter_details:
|
||||||
|
method = enc.method.name
|
||||||
|
if method not in INCLUDED_METHODS:
|
||||||
|
continue
|
||||||
|
|
||||||
|
encounters.append({
|
||||||
|
"pokemon_name": pokemon_name,
|
||||||
|
"national_dex": dex_num,
|
||||||
|
"method": method,
|
||||||
|
"chance": enc.chance,
|
||||||
|
"min_level": enc.min_level,
|
||||||
|
"max_level": enc.max_level,
|
||||||
|
})
|
||||||
|
|
||||||
|
return encounters
|
||||||
|
|
||||||
|
|
||||||
|
def aggregate_encounters(raw_encounters: list[dict]) -> list[dict]:
|
||||||
|
"""Aggregate encounter rates by pokemon + method (sum chances across level ranges)."""
|
||||||
|
agg: dict[tuple[int, str], dict] = {}
|
||||||
|
|
||||||
|
for enc in raw_encounters:
|
||||||
|
key = (enc["national_dex"], enc["method"])
|
||||||
|
if key not in agg:
|
||||||
|
agg[key] = {
|
||||||
|
"national_dex": enc["national_dex"],
|
||||||
|
"pokemon_name": enc["pokemon_name"],
|
||||||
|
"method": enc["method"],
|
||||||
|
"encounter_rate": 0,
|
||||||
|
"min_level": enc["min_level"],
|
||||||
|
"max_level": enc["max_level"],
|
||||||
|
}
|
||||||
|
agg[key]["encounter_rate"] += enc["chance"]
|
||||||
|
agg[key]["min_level"] = min(agg[key]["min_level"], enc["min_level"])
|
||||||
|
agg[key]["max_level"] = max(agg[key]["max_level"], enc["max_level"])
|
||||||
|
|
||||||
|
result = list(agg.values())
|
||||||
|
for r in result:
|
||||||
|
r["encounter_rate"] = min(r["encounter_rate"], 100)
|
||||||
|
|
||||||
|
return sorted(result, key=lambda x: (-x["encounter_rate"], x["pokemon_name"]))
|
||||||
|
|
||||||
|
|
||||||
|
def process_version(version_name: str, vg_info: dict) -> list[dict]:
|
||||||
|
"""Process all locations for a specific game version."""
|
||||||
|
print(f"\n--- Processing {version_name} ---")
|
||||||
|
|
||||||
|
region = pb.region(vg_info["region_id"])
|
||||||
|
location_refs = list(region.locations)
|
||||||
|
|
||||||
|
# For HGSS, also include Kanto locations
|
||||||
|
if version_name in ("heartgold", "soulsilver"):
|
||||||
|
kanto = pb.region(1)
|
||||||
|
location_refs = location_refs + list(kanto.locations)
|
||||||
|
|
||||||
|
print(f" Found {len(location_refs)} locations")
|
||||||
|
|
||||||
|
routes = []
|
||||||
|
order = 1
|
||||||
|
|
||||||
|
for loc_ref in location_refs:
|
||||||
|
loc_name = loc_ref.name
|
||||||
|
loc_id = int(loc_ref.url.rstrip("/").split("/")[-1])
|
||||||
|
display_name = clean_location_name(loc_name)
|
||||||
|
|
||||||
|
location = pb.location(loc_id)
|
||||||
|
areas = location.areas
|
||||||
|
if not areas:
|
||||||
|
continue
|
||||||
|
|
||||||
|
all_encounters: list[dict] = []
|
||||||
|
area_specific: dict[str, list[dict]] = {}
|
||||||
|
|
||||||
|
for area_ref in areas:
|
||||||
|
area_id = int(area_ref.url.rstrip("/").split("/")[-1])
|
||||||
|
area_slug = area_ref.name
|
||||||
|
area_suffix = clean_area_name(area_slug, loc_name)
|
||||||
|
|
||||||
|
encounters = get_encounters_for_area(area_id, version_name)
|
||||||
|
if not encounters:
|
||||||
|
continue
|
||||||
|
|
||||||
|
if area_suffix and len(areas) > 1:
|
||||||
|
area_specific[area_suffix] = encounters
|
||||||
|
else:
|
||||||
|
all_encounters.extend(encounters)
|
||||||
|
|
||||||
|
# Area-specific encounters become separate routes
|
||||||
|
if area_specific:
|
||||||
|
for area_suffix, area_encs in area_specific.items():
|
||||||
|
aggregated = aggregate_encounters(area_encs)
|
||||||
|
if aggregated:
|
||||||
|
route_name = f"{display_name} ({area_suffix})"
|
||||||
|
for enc in aggregated:
|
||||||
|
all_pokemon_dex.add(enc["national_dex"])
|
||||||
|
routes.append({
|
||||||
|
"name": route_name,
|
||||||
|
"order": order,
|
||||||
|
"encounters": aggregated,
|
||||||
|
})
|
||||||
|
order += 1
|
||||||
|
|
||||||
|
if all_encounters:
|
||||||
|
aggregated = aggregate_encounters(all_encounters)
|
||||||
|
if aggregated:
|
||||||
|
for enc in aggregated:
|
||||||
|
all_pokemon_dex.add(enc["national_dex"])
|
||||||
|
routes.append({
|
||||||
|
"name": display_name,
|
||||||
|
"order": order,
|
||||||
|
"encounters": aggregated,
|
||||||
|
})
|
||||||
|
order += 1
|
||||||
|
|
||||||
|
print(f" Routes with encounters: {len(routes)}")
|
||||||
|
total_enc = sum(len(r["encounters"]) for r in routes)
|
||||||
|
print(f" Total encounter entries: {total_enc}")
|
||||||
|
|
||||||
|
return routes
|
||||||
|
|
||||||
|
|
||||||
|
def fetch_pokemon_data(dex_numbers: set[int]) -> list[dict]:
|
||||||
|
"""Fetch Pokemon name/type data for all collected dex numbers."""
|
||||||
|
print(f"\n--- Fetching {len(dex_numbers)} Pokemon ---")
|
||||||
|
|
||||||
|
pokemon_list = []
|
||||||
|
dex_sorted = sorted(dex_numbers)
|
||||||
|
|
||||||
|
for i, dex in enumerate(dex_sorted, 1):
|
||||||
|
poke = pb.pokemon(dex)
|
||||||
|
types = [t.type.name for t in poke.types]
|
||||||
|
pokemon_list.append({
|
||||||
|
"national_dex": dex,
|
||||||
|
"name": poke.name.title().replace("-", " "),
|
||||||
|
"types": types,
|
||||||
|
"sprite_url": f"https://raw.githubusercontent.com/PokeAPI/sprites/master/sprites/pokemon/{dex}.png",
|
||||||
|
})
|
||||||
|
|
||||||
|
if i % 50 == 0 or i == len(dex_sorted):
|
||||||
|
print(f" Fetched {i}/{len(dex_sorted)}")
|
||||||
|
|
||||||
|
return sorted(pokemon_list, key=lambda x: x["national_dex"])
|
||||||
|
|
||||||
|
|
||||||
|
def write_json(filename: str, data):
|
||||||
|
path = DATA_DIR / filename
|
||||||
|
with open(path, "w") as f:
|
||||||
|
json.dump(data, f, indent=2)
|
||||||
|
print(f" -> {path}")
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
# Check for custom base URL
|
||||||
|
if "--base-url" in sys.argv:
|
||||||
|
idx = sys.argv.index("--base-url")
|
||||||
|
base_url = sys.argv[idx + 1]
|
||||||
|
pb_common.BASE_URL = base_url + "/api/v2"
|
||||||
|
print(f"Using custom PokeAPI: {base_url}")
|
||||||
|
else:
|
||||||
|
print("Using public PokeAPI (pokebase caches to disk after first fetch)")
|
||||||
|
|
||||||
|
DATA_DIR.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
# Build games.json
|
||||||
|
games = []
|
||||||
|
for vg_info in VERSION_GROUPS.values():
|
||||||
|
for game_info in vg_info["games"].values():
|
||||||
|
games.append({
|
||||||
|
"name": game_info["name"],
|
||||||
|
"slug": game_info["slug"],
|
||||||
|
"generation": vg_info["generation"],
|
||||||
|
"region": vg_info["region"],
|
||||||
|
"release_year": game_info["release_year"],
|
||||||
|
})
|
||||||
|
|
||||||
|
write_json("games.json", games)
|
||||||
|
print(f"Wrote {len(games)} games to games.json")
|
||||||
|
|
||||||
|
# Process each version
|
||||||
|
for vg_info in VERSION_GROUPS.values():
|
||||||
|
for ver_name in vg_info["versions"]:
|
||||||
|
routes = process_version(ver_name, vg_info)
|
||||||
|
write_json(f"{ver_name}.json", routes)
|
||||||
|
|
||||||
|
# Fetch all Pokemon data
|
||||||
|
pokemon = fetch_pokemon_data(all_pokemon_dex)
|
||||||
|
write_json("pokemon.json", pokemon)
|
||||||
|
print(f"\nWrote {len(pokemon)} Pokemon to pokemon.json")
|
||||||
|
|
||||||
|
print("\nDone! JSON files written to seeds/data/")
|
||||||
|
print("Review route ordering and curate as needed.")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
120
backend/src/app/seeds/loader.py
Normal file
120
backend/src/app/seeds/loader.py
Normal file
@@ -0,0 +1,120 @@
|
|||||||
|
"""Database upsert helpers for seed data."""
|
||||||
|
|
||||||
|
from sqlalchemy import select
|
||||||
|
from sqlalchemy.dialects.postgresql import insert
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncSession
|
||||||
|
|
||||||
|
from app.models.game import Game
|
||||||
|
from app.models.pokemon import Pokemon
|
||||||
|
from app.models.route import Route
|
||||||
|
from app.models.route_encounter import RouteEncounter
|
||||||
|
|
||||||
|
|
||||||
|
async def upsert_games(session: AsyncSession, games: list[dict]) -> dict[str, int]:
|
||||||
|
"""Upsert game records, return {slug: id} mapping."""
|
||||||
|
for game in games:
|
||||||
|
stmt = insert(Game).values(
|
||||||
|
name=game["name"],
|
||||||
|
slug=game["slug"],
|
||||||
|
generation=game["generation"],
|
||||||
|
region=game["region"],
|
||||||
|
release_year=game.get("release_year"),
|
||||||
|
).on_conflict_do_update(
|
||||||
|
index_elements=["slug"],
|
||||||
|
set_={
|
||||||
|
"name": game["name"],
|
||||||
|
"generation": game["generation"],
|
||||||
|
"region": game["region"],
|
||||||
|
"release_year": game.get("release_year"),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
await session.execute(stmt)
|
||||||
|
|
||||||
|
await session.flush()
|
||||||
|
|
||||||
|
result = await session.execute(select(Game.slug, Game.id))
|
||||||
|
return {row.slug: row.id for row in result}
|
||||||
|
|
||||||
|
|
||||||
|
async def upsert_pokemon(session: AsyncSession, pokemon_list: list[dict]) -> dict[int, int]:
|
||||||
|
"""Upsert pokemon records, return {national_dex: id} mapping."""
|
||||||
|
for poke in pokemon_list:
|
||||||
|
stmt = insert(Pokemon).values(
|
||||||
|
national_dex=poke["national_dex"],
|
||||||
|
name=poke["name"],
|
||||||
|
types=poke["types"],
|
||||||
|
sprite_url=poke.get("sprite_url"),
|
||||||
|
).on_conflict_do_update(
|
||||||
|
index_elements=["national_dex"],
|
||||||
|
set_={
|
||||||
|
"name": poke["name"],
|
||||||
|
"types": poke["types"],
|
||||||
|
"sprite_url": poke.get("sprite_url"),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
await session.execute(stmt)
|
||||||
|
|
||||||
|
await session.flush()
|
||||||
|
|
||||||
|
result = await session.execute(select(Pokemon.national_dex, Pokemon.id))
|
||||||
|
return {row.national_dex: row.id for row in result}
|
||||||
|
|
||||||
|
|
||||||
|
async def upsert_routes(
|
||||||
|
session: AsyncSession,
|
||||||
|
game_id: int,
|
||||||
|
routes: list[dict],
|
||||||
|
) -> dict[str, int]:
|
||||||
|
"""Upsert route records for a game, return {name: id} mapping."""
|
||||||
|
for route in routes:
|
||||||
|
stmt = insert(Route).values(
|
||||||
|
name=route["name"],
|
||||||
|
game_id=game_id,
|
||||||
|
order=route["order"],
|
||||||
|
).on_conflict_do_update(
|
||||||
|
constraint="uq_routes_game_name",
|
||||||
|
set_={"order": route["order"]},
|
||||||
|
)
|
||||||
|
await session.execute(stmt)
|
||||||
|
|
||||||
|
await session.flush()
|
||||||
|
|
||||||
|
result = await session.execute(
|
||||||
|
select(Route.name, Route.id).where(Route.game_id == game_id)
|
||||||
|
)
|
||||||
|
return {row.name: row.id for row in result}
|
||||||
|
|
||||||
|
|
||||||
|
async def upsert_route_encounters(
|
||||||
|
session: AsyncSession,
|
||||||
|
route_id: int,
|
||||||
|
encounters: list[dict],
|
||||||
|
dex_to_id: dict[int, int],
|
||||||
|
) -> int:
|
||||||
|
"""Upsert encounters for a route, return count of upserted rows."""
|
||||||
|
count = 0
|
||||||
|
for enc in encounters:
|
||||||
|
pokemon_id = dex_to_id.get(enc["national_dex"])
|
||||||
|
if pokemon_id is None:
|
||||||
|
print(f" Warning: no pokemon_id for dex {enc['national_dex']}")
|
||||||
|
continue
|
||||||
|
|
||||||
|
stmt = insert(RouteEncounter).values(
|
||||||
|
route_id=route_id,
|
||||||
|
pokemon_id=pokemon_id,
|
||||||
|
encounter_method=enc["method"],
|
||||||
|
encounter_rate=enc["encounter_rate"],
|
||||||
|
min_level=enc["min_level"],
|
||||||
|
max_level=enc["max_level"],
|
||||||
|
).on_conflict_do_update(
|
||||||
|
constraint="uq_route_pokemon_method",
|
||||||
|
set_={
|
||||||
|
"encounter_rate": enc["encounter_rate"],
|
||||||
|
"min_level": enc["min_level"],
|
||||||
|
"max_level": enc["max_level"],
|
||||||
|
},
|
||||||
|
)
|
||||||
|
await session.execute(stmt)
|
||||||
|
count += 1
|
||||||
|
|
||||||
|
return count
|
||||||
120
backend/src/app/seeds/run.py
Normal file
120
backend/src/app/seeds/run.py
Normal file
@@ -0,0 +1,120 @@
|
|||||||
|
"""Seed runner — reads JSON files and upserts into the database."""
|
||||||
|
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from sqlalchemy import func, select
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncSession
|
||||||
|
|
||||||
|
from app.core.database import async_session
|
||||||
|
from app.models.game import Game
|
||||||
|
from app.models.pokemon import Pokemon
|
||||||
|
from app.models.route import Route
|
||||||
|
from app.models.route_encounter import RouteEncounter
|
||||||
|
from app.seeds.loader import (
|
||||||
|
upsert_games,
|
||||||
|
upsert_pokemon,
|
||||||
|
upsert_route_encounters,
|
||||||
|
upsert_routes,
|
||||||
|
)
|
||||||
|
|
||||||
|
DATA_DIR = Path(__file__).parent / "data"
|
||||||
|
|
||||||
|
GAME_FILES = ["firered", "leafgreen", "emerald", "heartgold", "soulsilver"]
|
||||||
|
|
||||||
|
|
||||||
|
def load_json(filename: str) -> list[dict]:
|
||||||
|
path = DATA_DIR / filename
|
||||||
|
with open(path) as f:
|
||||||
|
return json.load(f)
|
||||||
|
|
||||||
|
|
||||||
|
async def seed():
|
||||||
|
"""Run the full seed process."""
|
||||||
|
print("Starting seed...")
|
||||||
|
|
||||||
|
async with async_session() as session:
|
||||||
|
async with session.begin():
|
||||||
|
# 1. Upsert games
|
||||||
|
games_data = load_json("games.json")
|
||||||
|
slug_to_id = await upsert_games(session, games_data)
|
||||||
|
print(f"Games: {len(slug_to_id)} upserted")
|
||||||
|
|
||||||
|
# 2. Upsert Pokemon
|
||||||
|
pokemon_data = load_json("pokemon.json")
|
||||||
|
dex_to_id = await upsert_pokemon(session, pokemon_data)
|
||||||
|
print(f"Pokemon: {len(dex_to_id)} upserted")
|
||||||
|
|
||||||
|
# 3. Per game: upsert routes and encounters
|
||||||
|
total_routes = 0
|
||||||
|
total_encounters = 0
|
||||||
|
|
||||||
|
for game_slug in GAME_FILES:
|
||||||
|
game_id = slug_to_id.get(game_slug)
|
||||||
|
if game_id is None:
|
||||||
|
print(f"Warning: game '{game_slug}' not found, skipping")
|
||||||
|
continue
|
||||||
|
|
||||||
|
routes_data = load_json(f"{game_slug}.json")
|
||||||
|
route_map = await upsert_routes(session, game_id, routes_data)
|
||||||
|
total_routes += len(route_map)
|
||||||
|
|
||||||
|
for route in routes_data:
|
||||||
|
route_id = route_map.get(route["name"])
|
||||||
|
if route_id is None:
|
||||||
|
print(f" Warning: route '{route['name']}' not found")
|
||||||
|
continue
|
||||||
|
|
||||||
|
enc_count = await upsert_route_encounters(
|
||||||
|
session, route_id, route["encounters"], dex_to_id
|
||||||
|
)
|
||||||
|
total_encounters += enc_count
|
||||||
|
|
||||||
|
print(f" {game_slug}: {len(route_map)} routes")
|
||||||
|
|
||||||
|
print(f"\nTotal routes: {total_routes}")
|
||||||
|
print(f"Total encounters: {total_encounters}")
|
||||||
|
|
||||||
|
print("Seed complete!")
|
||||||
|
|
||||||
|
|
||||||
|
async def verify():
|
||||||
|
"""Run post-seed verification checks."""
|
||||||
|
print("\n--- Verification ---")
|
||||||
|
|
||||||
|
async with async_session() as session:
|
||||||
|
# Overall counts
|
||||||
|
games_count = (await session.execute(select(func.count(Game.id)))).scalar()
|
||||||
|
pokemon_count = (await session.execute(select(func.count(Pokemon.id)))).scalar()
|
||||||
|
routes_count = (await session.execute(select(func.count(Route.id)))).scalar()
|
||||||
|
enc_count = (await session.execute(select(func.count(RouteEncounter.id)))).scalar()
|
||||||
|
|
||||||
|
print(f"Games: {games_count}")
|
||||||
|
print(f"Pokemon: {pokemon_count}")
|
||||||
|
print(f"Routes: {routes_count}")
|
||||||
|
print(f"Route Encounters: {enc_count}")
|
||||||
|
|
||||||
|
# Per-game breakdown
|
||||||
|
result = await session.execute(
|
||||||
|
select(Game.name, func.count(Route.id))
|
||||||
|
.join(Route, Route.game_id == Game.id)
|
||||||
|
.group_by(Game.name)
|
||||||
|
.order_by(Game.name)
|
||||||
|
)
|
||||||
|
print("\nRoutes per game:")
|
||||||
|
for row in result:
|
||||||
|
print(f" {row[0]}: {row[1]}")
|
||||||
|
|
||||||
|
# Per-game encounter counts
|
||||||
|
result = await session.execute(
|
||||||
|
select(Game.name, func.count(RouteEncounter.id))
|
||||||
|
.join(Route, Route.game_id == Game.id)
|
||||||
|
.join(RouteEncounter, RouteEncounter.route_id == Route.id)
|
||||||
|
.group_by(Game.name)
|
||||||
|
.order_by(Game.name)
|
||||||
|
)
|
||||||
|
print("\nEncounters per game:")
|
||||||
|
for row in result:
|
||||||
|
print(f" {row[0]}: {row[1]}")
|
||||||
|
|
||||||
|
print("\nVerification complete!")
|
||||||
@@ -7,6 +7,7 @@ services:
|
|||||||
- "8000:8000"
|
- "8000:8000"
|
||||||
volumes:
|
volumes:
|
||||||
- ./backend/src:/app/src:cached
|
- ./backend/src:/app/src:cached
|
||||||
|
- ./backend/alembic.ini:/app/alembic.ini:cached
|
||||||
environment:
|
environment:
|
||||||
- DEBUG=true
|
- DEBUG=true
|
||||||
- DATABASE_URL=postgresql://postgres:postgres@db:5432/nuzlocke
|
- DATABASE_URL=postgresql://postgres:postgres@db:5432/nuzlocke
|
||||||
|
|||||||
Reference in New Issue
Block a user