Spring 2025
Lepidoptica
Lepidoptica is the senior project that led me to graduation. The project instructed my team of four to find an instructor at the University of Florida (UF) to act as a our advisor, then craft and develop a program on their behalf. On our search for an advisor, we found Keith Willmott, instructor at UF and Curator of Lepidoptera at the Florida Museum of Natural History, McGuire Center for Lepidoptera & Biodiversity (basically he knows all about butterflies and moths).
We asked him what would be a project idea that his Center would find useful. He showed us the museum's database full of butterfly and moth specimen photos. Each photo included the butterfly itself, a color chart, the scientific name, gender, museum tag, and more notes specifying information about the specimen. Willmott's problem is that everything written in these thousands of photos need to be electronically documented in text writing, but it's a long, tedious process. Not only that, but a lot of the notes are handwritten and very old, making them illegible to human eyes.
My team developed a solution using a combination of a Python webapp and Easy OCR text model made in Google Colab that can begin to translate handwritten words into text and sort them into categories. The output can then be downloaded and imported into the museum's database for easy use.
What I Learned
I've created many machine learning models up to this point in my career, but I've never made a picture-text to computer-text model before. Through my research, I saw how difficult it is to specifically translate handwritten text into computer-text. But I wasn't deterred, Willmott emphasized how helpful it would be to have something that could even remotely understand handwritten text. So I came up with a solution. It was also new for me to use Flask and Python to create a webapp. Though I picked it up pretty fast.
What I Did
-
Fine-tuned EasyOCR through Google Colab, with 9243 images transcribed through Nanonets. The final model was 73% accurate and capable of recognizing handwriting, 33 new symbols (unique to this project's context), and better suited for our input images.
-
Integrated the fine-tuned model into the web-app.
-
Reformatted the image re-preprocessing page for easier use.
-
Had weekly meetings with Willmott to discuss project progress, demonstrate what we have going on, take criticism, and understand what kind of product he was looking for.
-
Presented our project to the rest of the class and all of the sponsors in an end-of-the-year project fair.
-
Packaged and delivered the project to Willmott so that progress can continue to be made to the program after our graduation.
Honorable Memory
Working with Willmott gave me a perspective of not only Entemology and butterfly research as a whole, but also the perspective of someone who doesn't know anything about programming or software development. He didn't know what was possible and what wasn't, he just told us the problem and we crafted a solution. It took a lot of communication, but it was all worth being able to deliver the best possible product.