Hardware Web UI

Overview

From November 2022 to January 2024, while I was an ECE student at SRMIST, I worked part-time as a Project Assistant on a breast cancer detection research project. It started under the Xilinx Women in Technology grant (Spring 2021) and got funding to move forward in late 2022. The research team had a neural network that sorts breast scans into BI-RADS categories. I built the software that let people actually use it: a web app and API server that ran on a PYNQ-Z2 board with the model.

The Problem

The PYNQ-Z2 has no user-facing interface. Out of the box, the only way to work with it is through Jupyter notebooks. That's fine for researchers, but it isn't something a medical reviewer can sit down and use.

The team needed a way to:

• Upload a scan and choose its imaging modality
• Run it through the model on the board
• Read the result clearly and print it
• Do all of this from another machine, without touching the board, even though the board had no public network address

My Role

We were a small team of 3–4: research faculty, professors and students. A research assistant trained the model and gave me the trained weights (the .h5 file). A designer worked on the UI design. I built everything between the model and the user:

• Frontend: the full React + TypeScript app, from upload to results
• Backend: a FastAPI server that ran on the board and handled uploads and inference requests
• Inference wrapper: model.py, which loads the weights, preprocesses images and maps predictions to BI-RADS categories
• Tooling: shell scripts to install dependencies, start the server, open the tunnel and ship new builds
• Handover: code documentation and cleanup in January 2024, when I handed the project over

How It Works

• Step 1: I built the React app on my machine and copied the production build to the board's filesystem. FastAPI served it as static files, so the board ran a single Python process on one port.
• Step 2: A reviewer opens the ngrok URL in their browser. ngrok forwards the traffic to the server on the board.
• Step 3: The reviewer picks a modality (Mammogram or Ultrasound & MRI) and uploads a scan. The server saves the image along with a small JSON record.
• Step 4: When they click Run model, the server resizes the image to 224×224 with OpenCV and runs it through the Keras model on the board's CPU.
• Step 5: The UI shows the predicted BI-RADS category (1–5), a short comment (from "Negative" up to "Highly Suggestive of Malignancy") and the model's confidence. The reviewer can print the result.

On the frontend I added zoom and pan controls for inspecting the scan, clear loading, success and error states for every step, and a print-friendly layout. The app only runs on desktop-sized screens, since small screens are too cramped for reading scans.

Challenges

• Python dependencies on the board: TensorFlow, Keras, FastAPI and Pydantic all had to be pinned to versions that installed and worked together on the board. I kept a separate, smaller dependency list just for the board.
• Reaching a board on a local network: For the proof of concept I used an ngrok tunnel to give the board a public URL. The long-term plan was a more permanent setup, such as Cloudflare Tunnel.
• Limited storage: Uploaded scans pile up quickly on a small board. A background thread checks every five minutes and deletes anything older than a day.
• Sharing without the hardware: I also deployed a UI-only demo on Netlify, so people could see the interface without access to the board. It has no model behind it.

Outcome

• A working end-to-end proof of concept: upload a scan in the browser and get a BI-RADS classification computed on the PYNQ-Z2, reachable from anywhere through a URL.
• Medical professionals reviewed it, and we demoed it at an internal expo between SRMIST's engineering and medical colleges.
• The research goal was to evaluate the model's accuracy and how much it could cut the time needed to analyze scans. The interface gave reviewers a way to run scans through the model without touching the board.
• I handed it over in January 2024 with documentation for the next people on the project.

What I Learned

• Running on real hardware is its own problem: Code that works on a laptop doesn't automatically work on an ARM board. Keeping dependency versions compatible was a big part of the work.
• Building for people outside the lab: Most of the value was in making the model usable for someone who would never open a notebook.
• Working around someone else's model: Treating the model as a black box with a clear contract (image in, category and confidence out) let the model and the interface change independently.
• Handover counts: Documenting the setup and scripts was what made the project usable after I left.