We introduce Widget2Code, the first image-only benchmark specifically designed to evaluate widget-to-code generation. Unlike existing UI2Code datasets that focus on web pages or mobile screens with accessible markup, widgets are compact, context-free micro-interfaces with dense layouts and strict spatial constraints, making code-paired data nearly impossible to collect. Our benchmark comprises 2,825 high-quality widgets curated from design platforms (Figma, Dribbble, Refero) and real device screenshots, with 1,000 samples reserved for rigorous testing.

To enable fine-grained evaluation beyond coarse global similarity metrics, we propose a suite of visual-only metrics inspired by Apple's Human Interface Guidelines. These metrics assess three critical dimensions: Layout (margin symmetry, content aspect ratio, area ratio), Legibility (text overlap, contrast consistency), and Style (palette fidelity, vibrancy, polarity). Our benchmark reveals that while generalized MLLMs (GPT-4o, Gemini-2.5) outperform specialized UI2Code models, all methods struggle with structural consistency, color accuracy, and dimension preservation—highlighting the unique challenges of widget reconstruction.

We benchmark two groups of works: (1) generalized MLLMs like GPT-4o, Gemini2.5-Pro, Seed1.6-Thinking, Qwen3-VL and Qwen3-VL-235b; (2) specialized UI2Code methods built based upon MLLMs, e.g., ScreenCoder, UI-UG, DCGen, UICopilot, LatCoder, Design2Code, and WebSight-VLM-8B.

Key Findings: Specialized UI2Code models, although effective on web and mobile datasets, exhibit pronounced performance degradation on widgets. In contrast, general-purpose MLLMs such as GPT-4o and Gemini achieve higher visual fidelity, suggesting better perceptual grounding, yet they still struggle to preserve structural consistency and stylistic accuracy. Moreover, all methods fail to reproduce the exact widget dimensions, even when explicitly prompted to match the input size.

Detailed quantitative results on our Widget2Code benchmark. The table shows performance across all evaluation metrics: Layout (Margin, Content, Area), Legibility (Text, Contrast, LocCon), Style (Palette, Vibrancy, Polarity), Perceptual (SSIM, LPIPS, CLIP), and Geometry. The best-performing model in each metric is in-bold, and the second best is underlined. Note that for LPIPS, lower values are better.

We present qualitative comparisons between our method (Widget2Code) and baseline approaches across diverse widget designs. Each example shows the input widget alongside outputs from generalized MLLMs (Gemini-2.5-Pro, GPT-4o, Qwen3-VL), specialized UI2Code models (ScreenCoder, UI-UG), and our method. Our approach demonstrates superior visual fidelity, structural consistency, and accurate color reproduction across various widget types.

To address the limitations exposed by our benchmark, we develop WidgetFactory, an end-to-end infrastructure that bridges perceptual understanding and executable code generation. At the perceptual level, we design a modular agent that follows widget design principles to decompose input images into atomic components—integrating icon retrieval from a 50k SVG library, reusable component templates for charts and buttons, and automated color palette extraction. This structured analysis prevents hallucination and ensures semantic consistency.

At the system level, WidgetFactory introduces WidgetDSL, a compact domain-specific language that encodes layouts, styles, and hierarchies in a framework-agnostic format. A deterministic compiler translates WidgetDSL into multiple front-end implementations (React, HTML/CSS), while an adaptive rendering module uses feedback-guided binary search to optimize dimensions, prevent overflow, and satisfy compactness constraints. This unified pipeline substantially enhances visual fidelity, achieving perfect geometry scores and establishing a strong baseline for future Widget2Code research.

For non-icon components, we define a library of reusable templates written in our DSL format. Each template encodes the structural and functional logic of common widget components such as buttons, charts, and text blocks, exposing configurable parameters for style, data binding, and runtime behavior. Given an extracted component, the system retrieves the corresponding component template and prompts the MLLM to refine or populate the template, producing a customized DSL instance that preserves the predefined component structure while adapting its visual style and data semantics to the input widget.

Below we showcase usage examples from our component library: BarChart, LineChart, PieChart, RadarChart, StackedBar, Sparkline, Button, Icon, ProgressBar, AppLogo, Text, and Checkbox. Each component type is demonstrated through multiple instantiations with varying visual styles and data configurations, illustrating how the same reusable template can be adapted to different design requirements.