Boss Digital


This real-world Pokédex identifies pocket monsters using ChatGPT and AI voice cloning

In context: A YouTuber recreated a real-life working version of the Pokédex from the 1990s anime series. For those unfamiliar, the Pokédex was a red camera-like device that Ash used to identify Pokémon. The gadget would analyze the creature and then vocally tell Ash its name and description.

Abe Haskins, the engineer and host behind the YouTube channel Abe’s Projects, wanted to recreate the Pokédex’s nostalgic functionality and appearance. Companies like Tiger have released Pokédex toys, but none worked like the one from the show. Abe’s main goals were to make it look as much like the anime version as possible, have it recognize most Pokémon in most situations, and it needed to be able to speak.

He succeeded on most points. The only thing it lacked was some buttons and components on the inside of the door flap, but to be fair, these appear to be cosmetic in the show. It would have added much more complexity with minimal payoff to an already “stressful” project.

Electronically, all Haskins needed was a camera sensor, a small screen to display the identified monster, a speaker for the voice, and WiFi so it could communicate with ChatGPT-4. Rounding out the hardware, he added a battery, a USB port, and a couple of buttons for scanning and exiting. He used a 3D printer to create the chassis, and the final design nicely matches the TV version down to the notched door.

Haskins admitted that the most challenging part of the project was designing and debugging the software.

Cloning the Pokédex’s voice (actor Nick Stellate) was easy using an online AI platform called PlayHT. The text-to-voice software struggled with the proper pronunciation of some pocket monster names. However, it worked well enough.

The device feeds camera-sensor data to ChatGPT-4, which compares it against a database of Pokémon images and information via an API. The database contains bitmap images paired with Pokémon names. The pictures render well on the small monochrome OLED panel, but the text sometimes appears garbled.

Still, all assembled, the Pokédex looked and worked as advertised, so it was time to debug. Haskins said debugging took most of his time, but five glitches were the most annoying and the hardest to fix.

  1. Loading the Pokémon bitmap images crashed the device. He fixed this by rewriting the code to read the bitmap byte by byte and rendering it pixel by pixel.
  2. An audible ticking sound was heard when the Pokédex spoke. A hex editor revealed “1000” at regular intervals throughout the API data. Manually removing all instances cleared up the issue.
  3. Splitting the Pokémon name from its description crashed the device. A misconfigured buffer was the culprit causing a buffer overflow.
  4. PSRAM appeared to trigger random crashes. Haskins had turned the PSRAM on earlier in the debugging process to fix an unrelated issue and forgot to disable it.
  5. Finally, the Pokédex occasionally freezes at random. The YouTuber couldn’t pinpoint or replicate the problem, so as of project completion, that’s the only glitch remaining.

Haskins explained that he could fix the persistent crashing issue by recoding the program but indicated he didn’t have time.

“I would love it if I could have time to rewrite the code and kind of start part of it over,” Haskins said. “But I think that’s the nature of these projects. They’re not always going to be exactly perfect.”

He tested the Pokédex out on a couple of toys. It could not correctly identify a plushie but recognized action figures and images on a computer screen quite well. Haskins says it’s likely because the stuffed Pokémon doesn’t look enough like its image from the database. One might imagine a similar success rate for a program designed to identify celebrities by comparing caricatures to photos.

Source link

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top