top of page
Training Gemini to Code
In collaboration with Google eng, we designed and built 12 unique apps, each with their own design system. These were then used to train Gemini to code from Ui screens.
Client
Year
2025
Role
Design Lead
Deliverables
Ui Screens
Jetpack Compose Code
Client Ask
Design 13 unique apps each with it's own design systems and develop the associated code based exactly on the designs. Package this as data sets to Android, to train the Gemini Model to learn to code from the data sets.
How does Gemini learn to code?
The Google team used reinforcement learning to train the model to recreate the screen designs based on successfully replicating the code. I worked with IBM engineers to help create that data.
Strategy
We cast a wide net and identified key categories that would afford distinct visual identities and experience requirements.
Achieving uniqueness with 2 design libraries
Salesforce's Lightening Design System
To ensure truly unique app screens, each app needed it's own design library. We used Google's Material Theme Builder to generate accessible unique styles, which could be packaged for developer.
Android's Material3 & Material3 Expressive (new)
Lightening required the use of Tokens Studio, a plugin for Figma, to achieve the use of web components for Android, as well as the color and typogrophy styles established within the library.
Building
We designed and developed an app each week. After designs were finalized the dev team would pick it up as we began designing the next app. The margin for error was extremely slim, as any design review or QA issues would the progress of the next app.
The final app designs
Pixel perfect designs for our developers to create flawless code for Android to give to Gemini.
Client reception
Our work was featured on Android's blog
"To accelerate the start of your UI development process, you can now generate Compose code directly from a design mock. In a file without an existing preview, click Generate Code From Screenshot in the Preview panel. Gemini will use the provided image to generate a starting implementation, saving you from writing boilerplate code from scratch."
This was the first ever collaboration between the Android team and IBM. The work was very well received, and the work is being used across Google, beyond the team we were creating the assets for. It is the beginning of a winning partnership!
bottom of page