Add game data seeding from PokeAPI with level ranges
Seed the database with Pokemon game data for 5 games (FireRed, LeafGreen, Emerald, HeartGold, SoulSilver) using pokebase. Includes Alembic migrations for route unique constraints and encounter level ranges, a two-phase seed system (offline fetch to JSON, then idempotent upserts), and Dockerfile updates for the seed runner. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,11 @@
|
|||||||
|
---
|
||||||
|
# nuzlocke-tracker-igl3
|
||||||
|
title: Name Generation
|
||||||
|
status: draft
|
||||||
|
type: feature
|
||||||
|
priority: normal
|
||||||
|
created_at: 2026-02-05T13:45:15Z
|
||||||
|
updated_at: 2026-02-05T13:46:30Z
|
||||||
|
---
|
||||||
|
|
||||||
|
For nuzlockes I want to implement name generation. The user should be able to provide a naming scheme or a list of nick names that can then be selected when a new encounter is registered.
|
||||||
@@ -0,0 +1,26 @@
|
|||||||
|
---
|
||||||
|
# nuzlocke-tracker-j28y
|
||||||
|
title: Curate route ordering to match game progression
|
||||||
|
status: todo
|
||||||
|
type: task
|
||||||
|
created_at: 2026-02-05T13:37:39Z
|
||||||
|
updated_at: 2026-02-05T13:37:39Z
|
||||||
|
parent: nuzlocke-tracker-f5ob
|
||||||
|
---
|
||||||
|
|
||||||
|
Routes are currently in alphabetical order from PokeAPI. Update the order field in each game's JSON seed file to reflect actual game progression (e.g., Pallet Town → Route 1 → Viridian City → Route 2 → ...).
|
||||||
|
|
||||||
|
## Details
|
||||||
|
- 646 routes total across 5 games
|
||||||
|
- FireRed and LeafGreen share the same route progression (Kanto)
|
||||||
|
- HeartGold and SoulSilver share the same route progression (Johto + Kanto)
|
||||||
|
- Emerald has its own progression (Hoenn)
|
||||||
|
- So effectively 3 unique orderings to define
|
||||||
|
- After updating JSON files, re-run the seed: `podman compose exec -e PYTHONUNBUFFERED=1 -w /app/src api python -m app.seeds`
|
||||||
|
|
||||||
|
## Files
|
||||||
|
- `backend/src/app/seeds/data/firered.json`
|
||||||
|
- `backend/src/app/seeds/data/leafgreen.json`
|
||||||
|
- `backend/src/app/seeds/data/emerald.json`
|
||||||
|
- `backend/src/app/seeds/data/heartgold.json`
|
||||||
|
- `backend/src/app/seeds/data/soulsilver.json`
|
||||||
@@ -1,31 +1,34 @@
|
|||||||
---
|
---
|
||||||
# nuzlocke-tracker-k5lm
|
# nuzlocke-tracker-k5lm
|
||||||
title: Initial Game Data Seeding
|
title: Initial Game Data Seeding
|
||||||
status: todo
|
status: completed
|
||||||
type: task
|
type: task
|
||||||
priority: normal
|
priority: normal
|
||||||
created_at: 2026-02-04T15:44:12Z
|
created_at: 2026-02-04T15:44:12Z
|
||||||
updated_at: 2026-02-04T15:47:29Z
|
updated_at: 2026-02-05T13:37:50Z
|
||||||
parent: nuzlocke-tracker-f5ob
|
parent: nuzlocke-tracker-f5ob
|
||||||
---
|
---
|
||||||
|
|
||||||
Create seed data for the database with initial games, routes, and Pokémon.
|
Create seed data for the database with initial games, routes, and Pokémon.
|
||||||
|
|
||||||
## Checklist
|
## Checklist
|
||||||
- [ ] Research and compile data for MVP games:
|
- [x] Research and compile data for MVP games:
|
||||||
- [ ] Pokémon FireRed/LeafGreen (Gen 3 Kanto remakes - popular for Nuzlockes)
|
- [x] Pokémon FireRed/LeafGreen (Gen 3 Kanto remakes - popular for Nuzlockes)
|
||||||
- [ ] Pokémon Emerald (Gen 3 Hoenn)
|
- [x] Pokémon Emerald (Gen 3 Hoenn)
|
||||||
- [ ] Pokémon HeartGold/SoulSilver (Gen 4 Johto remakes)
|
- [x] Pokémon HeartGold/SoulSilver (Gen 4 Johto remakes)
|
||||||
- [ ] For each game, gather:
|
- [x] For each game, gather:
|
||||||
- [ ] All routes/areas in progression order
|
- [x] All routes/areas in progression order
|
||||||
- [ ] Available wild Pokémon per route
|
- [x] Available wild Pokémon per route
|
||||||
- [ ] Encounter methods (grass, surf, fish, etc.)
|
- [x] Encounter methods (grass, surf, fish, etc.)
|
||||||
- [ ] Create seed scripts/migrations to populate database
|
- [x] Create seed scripts/migrations to populate database
|
||||||
- [ ] Include Pokémon base data (national dex, names, types, sprite URLs)
|
- [x] Include Pokémon base data (national dex, names, types, sprite URLs)
|
||||||
- [ ] Document data sources for attribution
|
- [x] Document data sources for attribution
|
||||||
|
- [x] Curate route ordering to match game progression — split to nuzlocke-tracker-j28y
|
||||||
|
|
||||||
## Notes
|
## Notes
|
||||||
- Use PokeAPI or Bulbapedia as data sources
|
|
||||||
- Admin panel allows adding more games later
|
- Admin panel allows adding more games later
|
||||||
- Focus on accuracy for the 3 MVP games
|
- Focus on accuracy for the 3 MVP games
|
||||||
- Sprite URLs can point to existing sprite repositories
|
|
||||||
|
## Data Sources
|
||||||
|
- Game data (routes, encounters, Pokemon): [PokeAPI](https://pokeapi.co/) via [pokebase](https://github.com/PokeAPI/pokebase) Python wrapper
|
||||||
|
- Sprites: [PokeAPI/sprites](https://github.com/PokeAPI/sprites) on GitHub
|
||||||
+1
-1
@@ -9,7 +9,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
|
|||||||
&& rm -rf /var/lib/apt/lists/*
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
# Install Python dependencies
|
# Install Python dependencies
|
||||||
COPY pyproject.toml README.md ./
|
COPY pyproject.toml README.md alembic.ini ./
|
||||||
COPY src/ ./src/
|
COPY src/ ./src/
|
||||||
|
|
||||||
RUN pip install --no-cache-dir -e .
|
RUN pip install --no-cache-dir -e .
|
||||||
|
|||||||
@@ -21,6 +21,7 @@ dev = [
|
|||||||
"pytest>=8.0.0",
|
"pytest>=8.0.0",
|
||||||
"pytest-asyncio>=0.25.0",
|
"pytest-asyncio>=0.25.0",
|
||||||
"httpx>=0.28.0",
|
"httpx>=0.28.0",
|
||||||
|
"pokebase>=1.4.0",
|
||||||
]
|
]
|
||||||
|
|
||||||
[build-system]
|
[build-system]
|
||||||
|
|||||||
+32
@@ -0,0 +1,32 @@
|
|||||||
|
"""add unique constraint routes game name
|
||||||
|
|
||||||
|
Revision ID: 694df688fb02
|
||||||
|
Revises: 03e5f186a9d5
|
||||||
|
Create Date: 2026-02-05 13:01:30.631978
|
||||||
|
|
||||||
|
"""
|
||||||
|
from typing import Sequence, Union
|
||||||
|
|
||||||
|
from alembic import op
|
||||||
|
import sqlalchemy as sa
|
||||||
|
|
||||||
|
|
||||||
|
# revision identifiers, used by Alembic.
|
||||||
|
revision: str = '694df688fb02'
|
||||||
|
down_revision: Union[str, Sequence[str], None] = '03e5f186a9d5'
|
||||||
|
branch_labels: Union[str, Sequence[str], None] = None
|
||||||
|
depends_on: Union[str, Sequence[str], None] = None
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
"""Upgrade schema."""
|
||||||
|
# ### commands auto generated by Alembic - please adjust! ###
|
||||||
|
op.create_unique_constraint('uq_routes_game_name', 'routes', ['game_id', 'name'])
|
||||||
|
# ### end Alembic commands ###
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
"""Downgrade schema."""
|
||||||
|
# ### commands auto generated by Alembic - please adjust! ###
|
||||||
|
op.drop_constraint('uq_routes_game_name', 'routes', type_='unique')
|
||||||
|
# ### end Alembic commands ###
|
||||||
@@ -0,0 +1,34 @@
|
|||||||
|
"""add level range to route encounters
|
||||||
|
|
||||||
|
Revision ID: 9afcbafe9888
|
||||||
|
Revises: 694df688fb02
|
||||||
|
Create Date: 2026-02-05 13:32:35.559499
|
||||||
|
|
||||||
|
"""
|
||||||
|
from typing import Sequence, Union
|
||||||
|
|
||||||
|
from alembic import op
|
||||||
|
import sqlalchemy as sa
|
||||||
|
|
||||||
|
|
||||||
|
# revision identifiers, used by Alembic.
|
||||||
|
revision: str = '9afcbafe9888'
|
||||||
|
down_revision: Union[str, Sequence[str], None] = '694df688fb02'
|
||||||
|
branch_labels: Union[str, Sequence[str], None] = None
|
||||||
|
depends_on: Union[str, Sequence[str], None] = None
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
"""Upgrade schema."""
|
||||||
|
op.add_column('route_encounters', sa.Column('min_level', sa.SmallInteger(), nullable=False, server_default='0'))
|
||||||
|
op.add_column('route_encounters', sa.Column('max_level', sa.SmallInteger(), nullable=False, server_default='0'))
|
||||||
|
op.alter_column('route_encounters', 'min_level', server_default=None)
|
||||||
|
op.alter_column('route_encounters', 'max_level', server_default=None)
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
"""Downgrade schema."""
|
||||||
|
# ### commands auto generated by Alembic - please adjust! ###
|
||||||
|
op.drop_column('route_encounters', 'max_level')
|
||||||
|
op.drop_column('route_encounters', 'min_level')
|
||||||
|
# ### end Alembic commands ###
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
from sqlalchemy import ForeignKey, SmallInteger, String
|
from sqlalchemy import ForeignKey, SmallInteger, String, UniqueConstraint
|
||||||
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||||
|
|
||||||
from app.core.database import Base
|
from app.core.database import Base
|
||||||
@@ -6,6 +6,9 @@ from app.core.database import Base
|
|||||||
|
|
||||||
class Route(Base):
|
class Route(Base):
|
||||||
__tablename__ = "routes"
|
__tablename__ = "routes"
|
||||||
|
__table_args__ = (
|
||||||
|
UniqueConstraint("game_id", "name", name="uq_routes_game_name"),
|
||||||
|
)
|
||||||
|
|
||||||
id: Mapped[int] = mapped_column(primary_key=True)
|
id: Mapped[int] = mapped_column(primary_key=True)
|
||||||
name: Mapped[str] = mapped_column(String(100))
|
name: Mapped[str] = mapped_column(String(100))
|
||||||
|
|||||||
@@ -17,6 +17,8 @@ class RouteEncounter(Base):
|
|||||||
pokemon_id: Mapped[int] = mapped_column(ForeignKey("pokemon.id"), index=True)
|
pokemon_id: Mapped[int] = mapped_column(ForeignKey("pokemon.id"), index=True)
|
||||||
encounter_method: Mapped[str] = mapped_column(String(30))
|
encounter_method: Mapped[str] = mapped_column(String(30))
|
||||||
encounter_rate: Mapped[int] = mapped_column(SmallInteger)
|
encounter_rate: Mapped[int] = mapped_column(SmallInteger)
|
||||||
|
min_level: Mapped[int] = mapped_column(SmallInteger)
|
||||||
|
max_level: Mapped[int] = mapped_column(SmallInteger)
|
||||||
|
|
||||||
route: Mapped["Route"] = relationship(back_populates="route_encounters")
|
route: Mapped["Route"] = relationship(back_populates="route_encounters")
|
||||||
pokemon: Mapped["Pokemon"] = relationship(back_populates="route_encounters")
|
pokemon: Mapped["Pokemon"] = relationship(back_populates="route_encounters")
|
||||||
|
|||||||
@@ -0,0 +1,21 @@
|
|||||||
|
"""Entry point for running seeds.
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
python -m app.seeds # Run seed
|
||||||
|
python -m app.seeds --verify # Run seed + verification
|
||||||
|
"""
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import sys
|
||||||
|
|
||||||
|
from app.seeds.run import seed, verify
|
||||||
|
|
||||||
|
|
||||||
|
async def main():
|
||||||
|
await seed()
|
||||||
|
if "--verify" in sys.argv:
|
||||||
|
await verify()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
asyncio.run(main())
|
||||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,37 @@
|
|||||||
|
[
|
||||||
|
{
|
||||||
|
"name": "Pokemon FireRed",
|
||||||
|
"slug": "firered",
|
||||||
|
"generation": 3,
|
||||||
|
"region": "kanto",
|
||||||
|
"release_year": 2004
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "Pokemon LeafGreen",
|
||||||
|
"slug": "leafgreen",
|
||||||
|
"generation": 3,
|
||||||
|
"region": "kanto",
|
||||||
|
"release_year": 2004
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "Pokemon Emerald",
|
||||||
|
"slug": "emerald",
|
||||||
|
"generation": 3,
|
||||||
|
"region": "hoenn",
|
||||||
|
"release_year": 2005
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "Pokemon HeartGold",
|
||||||
|
"slug": "heartgold",
|
||||||
|
"generation": 4,
|
||||||
|
"region": "johto",
|
||||||
|
"release_year": 2010
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "Pokemon SoulSilver",
|
||||||
|
"slug": "soulsilver",
|
||||||
|
"generation": 4,
|
||||||
|
"region": "johto",
|
||||||
|
"release_year": 2010
|
||||||
|
}
|
||||||
|
]
|
||||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,329 @@
|
|||||||
|
"""Fetch game data from PokeAPI and write static JSON seed files.
|
||||||
|
|
||||||
|
Uses pokebase which provides built-in file caching — first run fetches
|
||||||
|
from the API, subsequent runs are instant from disk cache.
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
# Against public PokeAPI (cached after first run):
|
||||||
|
podman compose exec -w /app/src api python -m app.seeds.fetch_pokeapi
|
||||||
|
|
||||||
|
# Against local PokeAPI (no rate limits):
|
||||||
|
podman compose exec -w /app/src api python -m app.seeds.fetch_pokeapi --base-url http://pokeapi-app:8000
|
||||||
|
"""
|
||||||
|
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pokebase as pb
|
||||||
|
import pokebase.common as pb_common
|
||||||
|
|
||||||
|
DATA_DIR = Path(__file__).parent / "data"
|
||||||
|
|
||||||
|
# Game definitions
|
||||||
|
VERSION_GROUPS = {
|
||||||
|
"firered-leafgreen": {
|
||||||
|
"versions": ["firered", "leafgreen"],
|
||||||
|
"generation": 3,
|
||||||
|
"region": "kanto",
|
||||||
|
"region_id": 1,
|
||||||
|
"games": {
|
||||||
|
"firered": {
|
||||||
|
"name": "Pokemon FireRed",
|
||||||
|
"slug": "firered",
|
||||||
|
"release_year": 2004,
|
||||||
|
},
|
||||||
|
"leafgreen": {
|
||||||
|
"name": "Pokemon LeafGreen",
|
||||||
|
"slug": "leafgreen",
|
||||||
|
"release_year": 2004,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
"emerald": {
|
||||||
|
"versions": ["emerald"],
|
||||||
|
"generation": 3,
|
||||||
|
"region": "hoenn",
|
||||||
|
"region_id": 3,
|
||||||
|
"games": {
|
||||||
|
"emerald": {
|
||||||
|
"name": "Pokemon Emerald",
|
||||||
|
"slug": "emerald",
|
||||||
|
"release_year": 2005,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
"heartgold-soulsilver": {
|
||||||
|
"versions": ["heartgold", "soulsilver"],
|
||||||
|
"generation": 4,
|
||||||
|
"region": "johto",
|
||||||
|
"region_id": 2,
|
||||||
|
"games": {
|
||||||
|
"heartgold": {
|
||||||
|
"name": "Pokemon HeartGold",
|
||||||
|
"slug": "heartgold",
|
||||||
|
"release_year": 2010,
|
||||||
|
},
|
||||||
|
"soulsilver": {
|
||||||
|
"name": "Pokemon SoulSilver",
|
||||||
|
"slug": "soulsilver",
|
||||||
|
"release_year": 2010,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
# Encounter methods to include (excludes gift, legendary-only, etc.)
|
||||||
|
INCLUDED_METHODS = {
|
||||||
|
"walk",
|
||||||
|
"surf",
|
||||||
|
"old-rod",
|
||||||
|
"good-rod",
|
||||||
|
"super-rod",
|
||||||
|
"rock-smash",
|
||||||
|
"headbutt",
|
||||||
|
}
|
||||||
|
|
||||||
|
# Collect all pokemon dex numbers across games
|
||||||
|
all_pokemon_dex: set[int] = set()
|
||||||
|
|
||||||
|
|
||||||
|
def clean_location_name(name: str) -> str:
|
||||||
|
"""Convert PokeAPI location slug to a clean display name.
|
||||||
|
|
||||||
|
e.g. 'kanto-route-1' -> 'Route 1'
|
||||||
|
'pallet-town' -> 'Pallet Town'
|
||||||
|
"""
|
||||||
|
for prefix in [
|
||||||
|
"kanto-", "johto-", "hoenn-", "sinnoh-",
|
||||||
|
"unova-", "kalos-", "alola-", "galar-",
|
||||||
|
]:
|
||||||
|
if name.startswith(prefix):
|
||||||
|
name = name[len(prefix):]
|
||||||
|
break
|
||||||
|
|
||||||
|
name = name.replace("-", " ").title()
|
||||||
|
name = re.sub(r"Route (\d+)", r"Route \1", name)
|
||||||
|
return name
|
||||||
|
|
||||||
|
|
||||||
|
def clean_area_name(area_name: str, location_name: str) -> str | None:
|
||||||
|
"""Extract meaningful area suffix, or None if it's the default area."""
|
||||||
|
if area_name.startswith(location_name):
|
||||||
|
suffix = area_name[len(location_name):].strip("-").strip()
|
||||||
|
if not suffix or suffix == "area":
|
||||||
|
return None
|
||||||
|
return suffix.replace("-", " ").title()
|
||||||
|
return area_name.replace("-", " ").title()
|
||||||
|
|
||||||
|
|
||||||
|
def get_encounters_for_area(area_id: int, version_name: str) -> list[dict]:
|
||||||
|
"""Get encounter data for a location area, filtered by version."""
|
||||||
|
area = pb.location_area(area_id)
|
||||||
|
encounters = []
|
||||||
|
|
||||||
|
for pe in area.pokemon_encounters:
|
||||||
|
pokemon_url = pe.pokemon.url
|
||||||
|
dex_num = int(pokemon_url.rstrip("/").split("/")[-1])
|
||||||
|
pokemon_name = pe.pokemon.name
|
||||||
|
|
||||||
|
for vd in pe.version_details:
|
||||||
|
if vd.version.name != version_name:
|
||||||
|
continue
|
||||||
|
|
||||||
|
for enc in vd.encounter_details:
|
||||||
|
method = enc.method.name
|
||||||
|
if method not in INCLUDED_METHODS:
|
||||||
|
continue
|
||||||
|
|
||||||
|
encounters.append({
|
||||||
|
"pokemon_name": pokemon_name,
|
||||||
|
"national_dex": dex_num,
|
||||||
|
"method": method,
|
||||||
|
"chance": enc.chance,
|
||||||
|
"min_level": enc.min_level,
|
||||||
|
"max_level": enc.max_level,
|
||||||
|
})
|
||||||
|
|
||||||
|
return encounters
|
||||||
|
|
||||||
|
|
||||||
|
def aggregate_encounters(raw_encounters: list[dict]) -> list[dict]:
|
||||||
|
"""Aggregate encounter rates by pokemon + method (sum chances across level ranges)."""
|
||||||
|
agg: dict[tuple[int, str], dict] = {}
|
||||||
|
|
||||||
|
for enc in raw_encounters:
|
||||||
|
key = (enc["national_dex"], enc["method"])
|
||||||
|
if key not in agg:
|
||||||
|
agg[key] = {
|
||||||
|
"national_dex": enc["national_dex"],
|
||||||
|
"pokemon_name": enc["pokemon_name"],
|
||||||
|
"method": enc["method"],
|
||||||
|
"encounter_rate": 0,
|
||||||
|
"min_level": enc["min_level"],
|
||||||
|
"max_level": enc["max_level"],
|
||||||
|
}
|
||||||
|
agg[key]["encounter_rate"] += enc["chance"]
|
||||||
|
agg[key]["min_level"] = min(agg[key]["min_level"], enc["min_level"])
|
||||||
|
agg[key]["max_level"] = max(agg[key]["max_level"], enc["max_level"])
|
||||||
|
|
||||||
|
result = list(agg.values())
|
||||||
|
for r in result:
|
||||||
|
r["encounter_rate"] = min(r["encounter_rate"], 100)
|
||||||
|
|
||||||
|
return sorted(result, key=lambda x: (-x["encounter_rate"], x["pokemon_name"]))
|
||||||
|
|
||||||
|
|
||||||
|
def process_version(version_name: str, vg_info: dict) -> list[dict]:
|
||||||
|
"""Process all locations for a specific game version."""
|
||||||
|
print(f"\n--- Processing {version_name} ---")
|
||||||
|
|
||||||
|
region = pb.region(vg_info["region_id"])
|
||||||
|
location_refs = list(region.locations)
|
||||||
|
|
||||||
|
# For HGSS, also include Kanto locations
|
||||||
|
if version_name in ("heartgold", "soulsilver"):
|
||||||
|
kanto = pb.region(1)
|
||||||
|
location_refs = location_refs + list(kanto.locations)
|
||||||
|
|
||||||
|
print(f" Found {len(location_refs)} locations")
|
||||||
|
|
||||||
|
routes = []
|
||||||
|
order = 1
|
||||||
|
|
||||||
|
for loc_ref in location_refs:
|
||||||
|
loc_name = loc_ref.name
|
||||||
|
loc_id = int(loc_ref.url.rstrip("/").split("/")[-1])
|
||||||
|
display_name = clean_location_name(loc_name)
|
||||||
|
|
||||||
|
location = pb.location(loc_id)
|
||||||
|
areas = location.areas
|
||||||
|
if not areas:
|
||||||
|
continue
|
||||||
|
|
||||||
|
all_encounters: list[dict] = []
|
||||||
|
area_specific: dict[str, list[dict]] = {}
|
||||||
|
|
||||||
|
for area_ref in areas:
|
||||||
|
area_id = int(area_ref.url.rstrip("/").split("/")[-1])
|
||||||
|
area_slug = area_ref.name
|
||||||
|
area_suffix = clean_area_name(area_slug, loc_name)
|
||||||
|
|
||||||
|
encounters = get_encounters_for_area(area_id, version_name)
|
||||||
|
if not encounters:
|
||||||
|
continue
|
||||||
|
|
||||||
|
if area_suffix and len(areas) > 1:
|
||||||
|
area_specific[area_suffix] = encounters
|
||||||
|
else:
|
||||||
|
all_encounters.extend(encounters)
|
||||||
|
|
||||||
|
# Area-specific encounters become separate routes
|
||||||
|
if area_specific:
|
||||||
|
for area_suffix, area_encs in area_specific.items():
|
||||||
|
aggregated = aggregate_encounters(area_encs)
|
||||||
|
if aggregated:
|
||||||
|
route_name = f"{display_name} ({area_suffix})"
|
||||||
|
for enc in aggregated:
|
||||||
|
all_pokemon_dex.add(enc["national_dex"])
|
||||||
|
routes.append({
|
||||||
|
"name": route_name,
|
||||||
|
"order": order,
|
||||||
|
"encounters": aggregated,
|
||||||
|
})
|
||||||
|
order += 1
|
||||||
|
|
||||||
|
if all_encounters:
|
||||||
|
aggregated = aggregate_encounters(all_encounters)
|
||||||
|
if aggregated:
|
||||||
|
for enc in aggregated:
|
||||||
|
all_pokemon_dex.add(enc["national_dex"])
|
||||||
|
routes.append({
|
||||||
|
"name": display_name,
|
||||||
|
"order": order,
|
||||||
|
"encounters": aggregated,
|
||||||
|
})
|
||||||
|
order += 1
|
||||||
|
|
||||||
|
print(f" Routes with encounters: {len(routes)}")
|
||||||
|
total_enc = sum(len(r["encounters"]) for r in routes)
|
||||||
|
print(f" Total encounter entries: {total_enc}")
|
||||||
|
|
||||||
|
return routes
|
||||||
|
|
||||||
|
|
||||||
|
def fetch_pokemon_data(dex_numbers: set[int]) -> list[dict]:
|
||||||
|
"""Fetch Pokemon name/type data for all collected dex numbers."""
|
||||||
|
print(f"\n--- Fetching {len(dex_numbers)} Pokemon ---")
|
||||||
|
|
||||||
|
pokemon_list = []
|
||||||
|
dex_sorted = sorted(dex_numbers)
|
||||||
|
|
||||||
|
for i, dex in enumerate(dex_sorted, 1):
|
||||||
|
poke = pb.pokemon(dex)
|
||||||
|
types = [t.type.name for t in poke.types]
|
||||||
|
pokemon_list.append({
|
||||||
|
"national_dex": dex,
|
||||||
|
"name": poke.name.title().replace("-", " "),
|
||||||
|
"types": types,
|
||||||
|
"sprite_url": f"https://raw.githubusercontent.com/PokeAPI/sprites/master/sprites/pokemon/{dex}.png",
|
||||||
|
})
|
||||||
|
|
||||||
|
if i % 50 == 0 or i == len(dex_sorted):
|
||||||
|
print(f" Fetched {i}/{len(dex_sorted)}")
|
||||||
|
|
||||||
|
return sorted(pokemon_list, key=lambda x: x["national_dex"])
|
||||||
|
|
||||||
|
|
||||||
|
def write_json(filename: str, data):
|
||||||
|
path = DATA_DIR / filename
|
||||||
|
with open(path, "w") as f:
|
||||||
|
json.dump(data, f, indent=2)
|
||||||
|
print(f" -> {path}")
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
# Check for custom base URL
|
||||||
|
if "--base-url" in sys.argv:
|
||||||
|
idx = sys.argv.index("--base-url")
|
||||||
|
base_url = sys.argv[idx + 1]
|
||||||
|
pb_common.BASE_URL = base_url + "/api/v2"
|
||||||
|
print(f"Using custom PokeAPI: {base_url}")
|
||||||
|
else:
|
||||||
|
print("Using public PokeAPI (pokebase caches to disk after first fetch)")
|
||||||
|
|
||||||
|
DATA_DIR.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
# Build games.json
|
||||||
|
games = []
|
||||||
|
for vg_info in VERSION_GROUPS.values():
|
||||||
|
for game_info in vg_info["games"].values():
|
||||||
|
games.append({
|
||||||
|
"name": game_info["name"],
|
||||||
|
"slug": game_info["slug"],
|
||||||
|
"generation": vg_info["generation"],
|
||||||
|
"region": vg_info["region"],
|
||||||
|
"release_year": game_info["release_year"],
|
||||||
|
})
|
||||||
|
|
||||||
|
write_json("games.json", games)
|
||||||
|
print(f"Wrote {len(games)} games to games.json")
|
||||||
|
|
||||||
|
# Process each version
|
||||||
|
for vg_info in VERSION_GROUPS.values():
|
||||||
|
for ver_name in vg_info["versions"]:
|
||||||
|
routes = process_version(ver_name, vg_info)
|
||||||
|
write_json(f"{ver_name}.json", routes)
|
||||||
|
|
||||||
|
# Fetch all Pokemon data
|
||||||
|
pokemon = fetch_pokemon_data(all_pokemon_dex)
|
||||||
|
write_json("pokemon.json", pokemon)
|
||||||
|
print(f"\nWrote {len(pokemon)} Pokemon to pokemon.json")
|
||||||
|
|
||||||
|
print("\nDone! JSON files written to seeds/data/")
|
||||||
|
print("Review route ordering and curate as needed.")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -0,0 +1,120 @@
|
|||||||
|
"""Database upsert helpers for seed data."""
|
||||||
|
|
||||||
|
from sqlalchemy import select
|
||||||
|
from sqlalchemy.dialects.postgresql import insert
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncSession
|
||||||
|
|
||||||
|
from app.models.game import Game
|
||||||
|
from app.models.pokemon import Pokemon
|
||||||
|
from app.models.route import Route
|
||||||
|
from app.models.route_encounter import RouteEncounter
|
||||||
|
|
||||||
|
|
||||||
|
async def upsert_games(session: AsyncSession, games: list[dict]) -> dict[str, int]:
|
||||||
|
"""Upsert game records, return {slug: id} mapping."""
|
||||||
|
for game in games:
|
||||||
|
stmt = insert(Game).values(
|
||||||
|
name=game["name"],
|
||||||
|
slug=game["slug"],
|
||||||
|
generation=game["generation"],
|
||||||
|
region=game["region"],
|
||||||
|
release_year=game.get("release_year"),
|
||||||
|
).on_conflict_do_update(
|
||||||
|
index_elements=["slug"],
|
||||||
|
set_={
|
||||||
|
"name": game["name"],
|
||||||
|
"generation": game["generation"],
|
||||||
|
"region": game["region"],
|
||||||
|
"release_year": game.get("release_year"),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
await session.execute(stmt)
|
||||||
|
|
||||||
|
await session.flush()
|
||||||
|
|
||||||
|
result = await session.execute(select(Game.slug, Game.id))
|
||||||
|
return {row.slug: row.id for row in result}
|
||||||
|
|
||||||
|
|
||||||
|
async def upsert_pokemon(session: AsyncSession, pokemon_list: list[dict]) -> dict[int, int]:
|
||||||
|
"""Upsert pokemon records, return {national_dex: id} mapping."""
|
||||||
|
for poke in pokemon_list:
|
||||||
|
stmt = insert(Pokemon).values(
|
||||||
|
national_dex=poke["national_dex"],
|
||||||
|
name=poke["name"],
|
||||||
|
types=poke["types"],
|
||||||
|
sprite_url=poke.get("sprite_url"),
|
||||||
|
).on_conflict_do_update(
|
||||||
|
index_elements=["national_dex"],
|
||||||
|
set_={
|
||||||
|
"name": poke["name"],
|
||||||
|
"types": poke["types"],
|
||||||
|
"sprite_url": poke.get("sprite_url"),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
await session.execute(stmt)
|
||||||
|
|
||||||
|
await session.flush()
|
||||||
|
|
||||||
|
result = await session.execute(select(Pokemon.national_dex, Pokemon.id))
|
||||||
|
return {row.national_dex: row.id for row in result}
|
||||||
|
|
||||||
|
|
||||||
|
async def upsert_routes(
|
||||||
|
session: AsyncSession,
|
||||||
|
game_id: int,
|
||||||
|
routes: list[dict],
|
||||||
|
) -> dict[str, int]:
|
||||||
|
"""Upsert route records for a game, return {name: id} mapping."""
|
||||||
|
for route in routes:
|
||||||
|
stmt = insert(Route).values(
|
||||||
|
name=route["name"],
|
||||||
|
game_id=game_id,
|
||||||
|
order=route["order"],
|
||||||
|
).on_conflict_do_update(
|
||||||
|
constraint="uq_routes_game_name",
|
||||||
|
set_={"order": route["order"]},
|
||||||
|
)
|
||||||
|
await session.execute(stmt)
|
||||||
|
|
||||||
|
await session.flush()
|
||||||
|
|
||||||
|
result = await session.execute(
|
||||||
|
select(Route.name, Route.id).where(Route.game_id == game_id)
|
||||||
|
)
|
||||||
|
return {row.name: row.id for row in result}
|
||||||
|
|
||||||
|
|
||||||
|
async def upsert_route_encounters(
|
||||||
|
session: AsyncSession,
|
||||||
|
route_id: int,
|
||||||
|
encounters: list[dict],
|
||||||
|
dex_to_id: dict[int, int],
|
||||||
|
) -> int:
|
||||||
|
"""Upsert encounters for a route, return count of upserted rows."""
|
||||||
|
count = 0
|
||||||
|
for enc in encounters:
|
||||||
|
pokemon_id = dex_to_id.get(enc["national_dex"])
|
||||||
|
if pokemon_id is None:
|
||||||
|
print(f" Warning: no pokemon_id for dex {enc['national_dex']}")
|
||||||
|
continue
|
||||||
|
|
||||||
|
stmt = insert(RouteEncounter).values(
|
||||||
|
route_id=route_id,
|
||||||
|
pokemon_id=pokemon_id,
|
||||||
|
encounter_method=enc["method"],
|
||||||
|
encounter_rate=enc["encounter_rate"],
|
||||||
|
min_level=enc["min_level"],
|
||||||
|
max_level=enc["max_level"],
|
||||||
|
).on_conflict_do_update(
|
||||||
|
constraint="uq_route_pokemon_method",
|
||||||
|
set_={
|
||||||
|
"encounter_rate": enc["encounter_rate"],
|
||||||
|
"min_level": enc["min_level"],
|
||||||
|
"max_level": enc["max_level"],
|
||||||
|
},
|
||||||
|
)
|
||||||
|
await session.execute(stmt)
|
||||||
|
count += 1
|
||||||
|
|
||||||
|
return count
|
||||||
@@ -0,0 +1,120 @@
|
|||||||
|
"""Seed runner — reads JSON files and upserts into the database."""
|
||||||
|
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from sqlalchemy import func, select
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncSession
|
||||||
|
|
||||||
|
from app.core.database import async_session
|
||||||
|
from app.models.game import Game
|
||||||
|
from app.models.pokemon import Pokemon
|
||||||
|
from app.models.route import Route
|
||||||
|
from app.models.route_encounter import RouteEncounter
|
||||||
|
from app.seeds.loader import (
|
||||||
|
upsert_games,
|
||||||
|
upsert_pokemon,
|
||||||
|
upsert_route_encounters,
|
||||||
|
upsert_routes,
|
||||||
|
)
|
||||||
|
|
||||||
|
DATA_DIR = Path(__file__).parent / "data"
|
||||||
|
|
||||||
|
GAME_FILES = ["firered", "leafgreen", "emerald", "heartgold", "soulsilver"]
|
||||||
|
|
||||||
|
|
||||||
|
def load_json(filename: str) -> list[dict]:
|
||||||
|
path = DATA_DIR / filename
|
||||||
|
with open(path) as f:
|
||||||
|
return json.load(f)
|
||||||
|
|
||||||
|
|
||||||
|
async def seed():
|
||||||
|
"""Run the full seed process."""
|
||||||
|
print("Starting seed...")
|
||||||
|
|
||||||
|
async with async_session() as session:
|
||||||
|
async with session.begin():
|
||||||
|
# 1. Upsert games
|
||||||
|
games_data = load_json("games.json")
|
||||||
|
slug_to_id = await upsert_games(session, games_data)
|
||||||
|
print(f"Games: {len(slug_to_id)} upserted")
|
||||||
|
|
||||||
|
# 2. Upsert Pokemon
|
||||||
|
pokemon_data = load_json("pokemon.json")
|
||||||
|
dex_to_id = await upsert_pokemon(session, pokemon_data)
|
||||||
|
print(f"Pokemon: {len(dex_to_id)} upserted")
|
||||||
|
|
||||||
|
# 3. Per game: upsert routes and encounters
|
||||||
|
total_routes = 0
|
||||||
|
total_encounters = 0
|
||||||
|
|
||||||
|
for game_slug in GAME_FILES:
|
||||||
|
game_id = slug_to_id.get(game_slug)
|
||||||
|
if game_id is None:
|
||||||
|
print(f"Warning: game '{game_slug}' not found, skipping")
|
||||||
|
continue
|
||||||
|
|
||||||
|
routes_data = load_json(f"{game_slug}.json")
|
||||||
|
route_map = await upsert_routes(session, game_id, routes_data)
|
||||||
|
total_routes += len(route_map)
|
||||||
|
|
||||||
|
for route in routes_data:
|
||||||
|
route_id = route_map.get(route["name"])
|
||||||
|
if route_id is None:
|
||||||
|
print(f" Warning: route '{route['name']}' not found")
|
||||||
|
continue
|
||||||
|
|
||||||
|
enc_count = await upsert_route_encounters(
|
||||||
|
session, route_id, route["encounters"], dex_to_id
|
||||||
|
)
|
||||||
|
total_encounters += enc_count
|
||||||
|
|
||||||
|
print(f" {game_slug}: {len(route_map)} routes")
|
||||||
|
|
||||||
|
print(f"\nTotal routes: {total_routes}")
|
||||||
|
print(f"Total encounters: {total_encounters}")
|
||||||
|
|
||||||
|
print("Seed complete!")
|
||||||
|
|
||||||
|
|
||||||
|
async def verify():
|
||||||
|
"""Run post-seed verification checks."""
|
||||||
|
print("\n--- Verification ---")
|
||||||
|
|
||||||
|
async with async_session() as session:
|
||||||
|
# Overall counts
|
||||||
|
games_count = (await session.execute(select(func.count(Game.id)))).scalar()
|
||||||
|
pokemon_count = (await session.execute(select(func.count(Pokemon.id)))).scalar()
|
||||||
|
routes_count = (await session.execute(select(func.count(Route.id)))).scalar()
|
||||||
|
enc_count = (await session.execute(select(func.count(RouteEncounter.id)))).scalar()
|
||||||
|
|
||||||
|
print(f"Games: {games_count}")
|
||||||
|
print(f"Pokemon: {pokemon_count}")
|
||||||
|
print(f"Routes: {routes_count}")
|
||||||
|
print(f"Route Encounters: {enc_count}")
|
||||||
|
|
||||||
|
# Per-game breakdown
|
||||||
|
result = await session.execute(
|
||||||
|
select(Game.name, func.count(Route.id))
|
||||||
|
.join(Route, Route.game_id == Game.id)
|
||||||
|
.group_by(Game.name)
|
||||||
|
.order_by(Game.name)
|
||||||
|
)
|
||||||
|
print("\nRoutes per game:")
|
||||||
|
for row in result:
|
||||||
|
print(f" {row[0]}: {row[1]}")
|
||||||
|
|
||||||
|
# Per-game encounter counts
|
||||||
|
result = await session.execute(
|
||||||
|
select(Game.name, func.count(RouteEncounter.id))
|
||||||
|
.join(Route, Route.game_id == Game.id)
|
||||||
|
.join(RouteEncounter, RouteEncounter.route_id == Route.id)
|
||||||
|
.group_by(Game.name)
|
||||||
|
.order_by(Game.name)
|
||||||
|
)
|
||||||
|
print("\nEncounters per game:")
|
||||||
|
for row in result:
|
||||||
|
print(f" {row[0]}: {row[1]}")
|
||||||
|
|
||||||
|
print("\nVerification complete!")
|
||||||
@@ -7,6 +7,7 @@ services:
|
|||||||
- "8000:8000"
|
- "8000:8000"
|
||||||
volumes:
|
volumes:
|
||||||
- ./backend/src:/app/src:cached
|
- ./backend/src:/app/src:cached
|
||||||
|
- ./backend/alembic.ini:/app/alembic.ini:cached
|
||||||
environment:
|
environment:
|
||||||
- DEBUG=true
|
- DEBUG=true
|
||||||
- DATABASE_URL=postgresql://postgres:postgres@db:5432/nuzlocke
|
- DATABASE_URL=postgresql://postgres:postgres@db:5432/nuzlocke
|
||||||
|
|||||||
Reference in New Issue
Block a user