Skip to content

Load and scroll long tables smoothly: lazy DataTable and DataTable2 rows, no compositing layer per control - #7076

Open
FeodorFitsner wants to merge 14 commits into
mainfrom
feature/datatable-repaint
Open

FeodorFitsner wants to merge 14 commits into
mainfrom
feature/datatable-repaint

Conversation

@FeodorFitsner

@FeodorFitsner FeodorFitsner commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Description

A web app with a long DataTable (3,470 rows × 9 columns, about 35k controls) freezes for about 4 s when the table loads, freezes again for about a second whenever a row is selected, and stutters on every scroll. Embedded in Flet Studio's preview, it slows all of Studio. The scrolling stutter comes from how every control is wrapped. The freezes come from DataTable itself, which builds, measures and lays out every cell.

Cause

Since #6928, LayoutControl and BaseControl wrap every control in Opacity, including at the default 1.0, so adding or changing opacity doesn't rebuild the subtree. The comment there says Opacity at 1.0 paints its child without a layer. That's no longer true: Flutter's RenderOpacity is a repaint boundary (isRepaintBoundary => alwaysNeedsCompositing, which is child != null && alpha > 0). It has its own OpacityLayer at any opacity above 0, 1.0 included.

So every control was its own layer with its own picture. On web, each frame walks the whole layer tree (measureChildren, visitPicture, Picture.cullRect), off-screen layers included. In a CPU profile of a 30-step wheel scroll, that walk was about two-thirds of all main-thread time.

Fix

  • _ControlOpacity replaces Opacity in _opacity(). The wrapper is still always there, so the tree keeps its shape, but it paints the child directly at 1.0, paints nothing at 0.0, and pushes an OpacityLayer only in between. This is how RenderOpacity worked before it became a repaint boundary. animate_opacity still uses AnimatedOpacity.
  • DataTableControl paints the table behind a RepaintBoundary. RenderTable.paint paints every row with no culling, so inside a scrolling parent each frame re-recorded the whole table. With the boundary, a scroll moves the recorded picture instead. In a plain Flutter repro (no Flet), this alone cut scroll frames over 50 ms from 40 to 0.
  • DataTable builds only the rows near the viewport when it has 100 or more rows of one height (the default) and its rows are all selectable or none. The rows go into Flutter DataTables of 10 rows each, with their heading row clipped off, stacked under a header DataTable. Only the chunks near the nearest vertical Scrollable's viewport are built, so the table still scrolls with its parent Column, as before; no API change. Rows, checkboxes, ink, sort arrows and lines are still drawn by Flutter's DataTable.
    • The tables share column widths through a TableColumnWidth that keeps each column's widest cell measured so far. An offstage table seeds them with each column's widest texts, picked by adding up per-character advances, so a long value far down the table sizes its column before it scrolls into view. Widths are measured afresh when the rows are replaced.
    • The header's select-all checkbox counts every row, including rows not built, and selecting all sends select_change to every row, as before.
    • Smaller tables, rows of varying height (data_row_max_height=float("inf")), and tables that mix selectable and unselectable rows still render as one DataTable.
  • DataTable2 (flet-datatable2) builds only the rows near the viewport when it has 100 or more rows, its heading row fixed (the default) and no outer border. data_table_2 builds every row into one Table, so a 3,470-row table froze the app for about 2.5 s on load. flet-datatable2 now carries its own copy of data_table_2 3.0.0's DataTable2 (BSD 3-Clause, LICENSE kept), committed unmodified first so the second commit shows only the change:
    • Data rows go into 10-row Tables that a ListView builds as they scroll into view, with exact row heights (specific_row_height included). Fixed left columns get their own ListView on the existing synced controller. Each chunk after the first strokes the line between rows above itself, centered on the boundary as Table does.
    • The per-row parts of build() (row decoration, data cell, row checkbox) became methods that both paths use, so the eager path is unchanged.
    • The typos hook skips the vendored copy, which keeps upstream's spelling.
  • DataTableControl reads horizontal_lines. It read "horizontal_lLines", so DataTable.horizontal_lines never had any effect.
  • The flet Flutter package is bumped to 1.1.3.
  • The macOS integration-test goldens for the handling_events, sortable_and_selectable and spacing DataTable examples are updated. Those examples set horizontal_lines, which now draws; the images differ only in the row lines, and match the CI runner's screenshots pixel for pixel.
  • CI: the Linux client builds retry apt-get update/install up to three times. In the Debian 10 container, apt also gives up on a stalled download after 30 s and retries failed files. Previously an archive mirror serving a truncated package hung the job until its 15-minute timeout.

Measurements

Loading the 3,470-row app (CPU profile, profile build):

Longest frame Build Layout
main 3,967 ms 1,210 ms 2,105 ms (1,374 ms of it sizing columns)
This PR 417 ms (157 ms frame work, the rest decoding the message) 87 ms 100 ms

The same app in Flet Studio's preview: the longest frame while the table loads drops from 3,867 ms to 217 ms. Selecting a row in a 3,470-row table with a checkbox column: frames over 50 ms per click drop from 3 (one of 950–1,333 ms) to none.

Screenshots of the 3,470-row app match main pixel for pixel at 1400 and 1920 px wide and at ten scroll positions down to the last row. A 500-row table with a checkbox column, sorting, grid lines, a rounded border, edit icons and hover color matches through hovering, selecting, sorting and scrolling, apart from a handful of anti-aliased pixels where lines cross (at most 12/255 off).

The same 3,470 rows in a DataTable2: the longest frame drops from 2,517–2,833 ms to 267 ms (75 ms of frame work). Screenshots match upstream DataTable2 pixel for pixel at ten scroll positions, at 1920 and 1000 px wide. A 500-row DataTable2 with a checkbox column, two fixed left columns, grid lines, sorting, hover color, bottom_margin and taller rows matches through hovering, selecting, sorting, select-all and both scroll directions, apart from 1–21/255 of anti-aliasing along chunk boundaries.

Scrolling:

Measured on the 3,470-row app in headless Chrome 157, scrolled with the mouse wheel.

CPU profile of a 30-step scroll, profile builds:

Frame work Raster Main thread busy
main 5,957 ms 5,495 ms 7,630 ms
RepaintBoundary only 6,091 ms 5,872 ms 7,726 ms
This PR 47 ms 14 ms 1,309 ms

Frames longer than 50 ms during a 40-step scroll, release builds:

JS (CanvasKit) wasm (skwasm)
main 44 11–13
This PR 1–2 0–2

Before the lazy table, most of what was left was hit testing: RenderTable.hitTestChildren checks every cell on each pointer event, about 1 s over the 30 steps. The lazy table only hit-tests the rows it built; with it, the same 40-step scroll has no frames over 50 ms either (longest 33–50 ms).

Test code

3,000 rows in a scrolling Column, plus a box with buttons that set its opacity to 1.0, 0.4 and 0.0. Run it with flet run --web on 1.1.2 and on this branch, then scroll with the mouse wheel or drag the scrollbar. On 1.1.2 the page freezes for a few seconds while the table appears, and then stutters as it scrolls; on this branch the table appears at once and scrolls smoothly. The opacity buttons should show the box solid, faded and hidden, and the table should have horizontal lines (missing on 1.1.2).

import flet as ft

ROWS = 3000
COLUMNS = ["ID", "Item", "Description", "Quantity", "Warehouse", "Bin", "Category"]


def main(page: ft.Page):
    page.title = "DataTable scroll test"

    box = ft.Container(width=40, height=40, bgcolor=ft.Colors.BLUE)

    def set_opacity(e):
        box.opacity = float(e.control.data)
        box.update()

    page.add(
        ft.Row(
            [
                ft.Text(f"{ROWS} rows. Scroll with the mouse wheel or drag the scrollbar."),
                box,
                *[
                    ft.Button(f"opacity {v}", data=v, on_click=set_opacity)
                    for v in ("1.0", "0.4", "0.0")
                ],
            ]
        ),
        ft.Column(
            expand=True,
            scroll=ft.ScrollMode.AUTO,
            controls=[
                ft.DataTable(
                    horizontal_lines=ft.BorderSide(1, ft.Colors.OUTLINE_VARIANT),
                    columns=[ft.DataColumn(ft.Text(c)) for c in COLUMNS],
                    rows=[
                        ft.DataRow(
                            cells=[
                                ft.DataCell(ft.Text(str(r))),
                                ft.DataCell(ft.Text(f"P-{r:05}")),
                                ft.DataCell(ft.Text(f"Replacement part number {r}")),
                                ft.DataCell(ft.Text(str(r % 17))),
                                ft.DataCell(ft.Text(f"WH-{r % 9:02}")),
                                ft.DataCell(ft.Text(f"{r % 40:02}A")),
                                ft.DataCell(ft.Text("PARTS")),
                            ]
                        )
                        for r in range(ROWS)
                    ],
                )
            ],
        ),
    )


ft.run(main)

With this app, 40 wheel steps in headless Chrome gave 45–46 frames over 50 ms on 1.1.2 and none on this branch.

For DataTable2, the same rows in a DataTable2 (pip install flet-datatable2). On 1.1.2 the page freezes for a couple of seconds before the table shows; on this branch it shows at once. The table scrolls itself, with its header fixed:

import flet as ft
import flet_datatable2 as fdt

ROWS = 3000
COLUMNS = ["ID", "Item", "Description", "Quantity", "Warehouse", "Bin", "Category"]


def main(page: ft.Page):
    page.title = "DataTable2 load test"
    page.add(
        fdt.DataTable2(
            expand=True,
            horizontal_lines=ft.BorderSide(1, ft.Colors.OUTLINE_VARIANT),
            columns=[fdt.DataColumn2(ft.Text(c)) for c in COLUMNS],
            rows=[
                fdt.DataRow2(
                    cells=[
                        ft.DataCell(ft.Text(str(r))),
                        ft.DataCell(ft.Text(f"P-{r:05}")),
                        ft.DataCell(ft.Text(f"Replacement part number {r}")),
                        ft.DataCell(ft.Text(str(r % 17))),
                        ft.DataCell(ft.Text(f"WH-{r % 9:02}")),
                        ft.DataCell(ft.Text(f"{r % 40:02}A")),
                        ft.DataCell(ft.Text("PARTS")),
                    ]
                )
                for r in range(ROWS)
            ],
        )
    )


ft.run(main)

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Checklist

  • I signed the CLA.
  • I have performed a self-review of my own code.
  • My code follows the style guidelines of this project.
  • I have commented my code, particularly in hard-to-understand areas.
  • My changes generate no new warnings.
  • New and existing tests pass locally with my changes.
  • I have made corresponding documentation changes, if applicable.
  • I have added changelog entries for user-facing changes, if applicable.
  • I have updated release guide pages and website/sidebars.yml for breaking changes, removals, and deprecations, if applicable.

Additional details

  • New tests:
    • test/widgets/control_opacity_test.dart checks for no layer at 1.0 or 0.0, an OpacityLayer at 0.5, and that the child's state survives opacity changes.
    • test/widgets/datatable_test.dart checks the RepaintBoundary, that horizontal_lines/vertical_lines reach the TableBorder, and for a long table: only rows near the viewport are built and sit where one table would put them; columns line up and fit a long text (plain or Text control) far down the table; the header checkbox selects rows not yet built and shows rows selected out of view. The full flet package suite passes (191 tests).
    • flet_datatable2/test/datatable2_test.dart (new) checks that a long DataTable2 builds only rows near the viewport, reaches its last row, keeps fixed left columns in line while scrolling, and that a short one is still one Table.
  • Docs: the DataTable and DataTable2 docstrings say when a table builds only its visible rows, and what that needs.
  • Opacity changes are still checked in the browser with the test app above: 1.0, 0.4 and 0.0 render as expected.

…pacity

Every control was wrapped in Opacity, also at the default 1.0, to keep the
tree's shape stable. Flutter's RenderOpacity is a repaint boundary with its
own OpacityLayer at any opacity above 0, so each control had a layer. A
3,470-row DataTable (about 35k controls) spent ~6 s of frame work in a
30-step scroll on web, walking those layers every frame.

_ControlOpacity keeps the always-present wrapper but paints its child
directly at 1.0, nothing at 0.0, and pushes an OpacityLayer only in between.
The same scroll now takes 47 ms of frame work.

DataTableControl paints the table behind a RepaintBoundary, so a scrolling
parent moves the recorded picture instead of repainting every row, and reads
horizontal_lines (it read "horizontal_lLines", so the lines never showed).

Bump the flet Flutter package to 1.1.3.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've reviewed your changes and they look great!


Sourcery is free for open source - if you like our reviews please consider sharing them ✨

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

Deploying flet-website-v2 with  Cloudflare Pages  Cloudflare Pages

Latest commit: 34599fb
Status: ✅  Deploy successful!
Preview URL: https://a68ca1c5.flet-website-v2.pages.dev
Branch Preview URL: https://feature-datatable-repaint.flet-website-v2.pages.dev

View logs

…iewport

A DataTable of 100+ rows of one fixed height renders as 10-row DataTable
chunks under a header DataTable, building only the chunks near the nearest
vertical Scrollable's viewport. Chunks share column widths through a
TableColumnWidth that keeps each column's widest measured cell, seeded by an
offstage table of each column's widest texts.

On a 3,470-row x 9-column table: the load frame drops from 3.4 s to 0.16 s,
and selecting a row from ~1.2 s to under 50 ms. Screenshots match one
DataTable pixel for pixel.
Unmodified copy of data_table_2's DataTable2 (BSD 3-Clause, LICENSE kept)
in place of the pub dependency, so the next commit can change how it
builds its rows. The SDK floor rises to 3.8 for the copy's null-aware
elements; Flutter 3.47 is already required. typos skips the copy, which
keeps upstream's spelling.
A DataTable2 of 100+ rows with its heading row fixed and no outer border
builds its data rows in 10-row Tables that a ListView creates as they
scroll into view, instead of one Table of every cell; fixed left columns
get their own ListView on the synced controller. Each chunk after the
first strokes the line between rows above itself, as Table does. The
per-row parts of build() move into methods both paths share.

On a 3,470-row x 9-column table the load frame drops from 2.5-2.8 s to
0.27 s. Screenshots match upstream DataTable2 pixel for pixel at ten
scroll positions; with fixed columns, checkboxes, lines and taller rows,
the only differences are 1-21/255 of anti-aliasing at chunk boundaries.
@FeodorFitsner FeodorFitsner changed the title Scroll long pages and tables smoothly: no compositing layer per control Load and scroll long tables smoothly: lazy DataTable and DataTable2 rows, no compositing layer per control Nov 12, 2026
The handling_events, sortable_and_selectable and spacing examples set
DataTable.horizontal_lines, which had no effect until this PR fixed the
"horizontal_lLines" key. Their goldens change only in the row lines; the
new images match the CI runner's screenshots pixel for pixel.
The Debian 10 build failed when an archive mirror served a truncated
package and apt then hung until the 15-minute step timeout. In the
Debian 10 container apt now gives up on a stalled download after 30 s
and retries failed files (Acquire::Retries 5), so it moves on to the
next mirror; apt-get update and install are retried up to three times
in every Linux container.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The lazy DataTable implementation can crash on zero-height rows, retain stale or inconsistent column widths, and expose invisible checkboxes with misleading actions to screen readers.

4 open issues
What changed in this PR

Improves long-table loading and scrolling in Flet by building rows near the viewport and removing unnecessary opacity layers, while preserving the stable control wrapper introduced in #6928.

Changes:

  • Adds lazy rendering for eligible DataTable and DataTable2 configurations, with shared column sizing and synchronized fixed columns.
  • Optimizes opacity painting and table repaints, and fixes DataTable.horizontal_lines.
  • Adds widget tests and documentation, bumps the Flutter package to 1.1.3, and adds Linux dependency-install retries.
File Description
sdk/​python/​packages/​flet/​src/​flet/​controls/​material/​datatable.py Documents lazy rendering requirements and column sizing.
sdk/​python/​packages/​flet-datatable2/​src/​flutter/​flet_datatable2/​test/​datatable2_test.dart Tests lazy rows, scrolling, fixed columns, and eager fallback.
sdk/​python/​packages/​flet-datatable2/​src/​flutter/​flet_datatable2/​pubspec.yaml Removes the upstream dependency and updates SDK/test requirements.
sdk/​python/​packages/​flet-datatable2/​src/​flutter/​flet_datatable2/​lib/​src/​utils/​datatable.dart Uses vendored column-size definitions.
sdk/​python/​packages/​flet-datatable2/​src/​flutter/​flet_datatable2/​lib/​src/​datatable2.dart Switches the control to the vendored implementation.
sdk/​python/​packages/​flet-datatable2/​src/​flutter/​flet_datatable2/​lib/​src/​data_table_2/​LICENSE Retains the upstream BSD license.
sdk/​python/​packages/​flet-datatable2/​src/​flutter/​flet_datatable2/​lib/​src/​data_table_2/​data_table_2.dart Adds vendored table rendering with lazy row chunks.
sdk/​python/​packages/​flet-datatable2/​src/​flutter/​flet_datatable2/​lib/​src/​data_table_2/​data_table_2_resizable.dart Includes upstream column-resizing support.
sdk/​python/​packages/​flet-datatable2/​src/​flet_datatable2/​datatable2.py Documents DataTable2 lazy-rendering requirements.
sdk/​python/​.pre-commit-config.yaml Excludes vendored source from spelling checks.
packages/​flet/​test/​widgets/​datatable_test.dart Tests rendering, borders, sizing, scrolling, and selection.
packages/​flet/​test/​widgets/​control_opacity_test.dart Tests opacity layers and child-state preservation.
packages/​flet/​pubspec.yaml Bumps the Flutter package to 1.1.3.
packages/​flet/​lib/​src/​controls/​datatable.dart Adds lazy chunks, shared widths, repaint boundaries, and horizontal-line parsing.
packages/​flet/​lib/​src/​controls/​base_controls.dart Avoids opacity layers at fully opaque and transparent values.
packages/​flet/​CHANGELOG.md Records Flutter rendering improvements and fixes.
client/​pubspec.lock Updates Flet and removes the upstream table dependency.
CHANGELOG.md Records user-facing performance improvements and the line fix.
.github/​workflows/​ci.yml Adds apt retries and Debian archive-download timeouts.

Review effort: Balanced

…t-out

- Follow the ScrollPosition itself, so a position a Scrollable swaps in
  keeps building rows as it scrolls.
- Keep the hidden rows of chunk and header tables (heading checkbox,
  stand-in rows) and the offstage seed table out of keyboard focus; a
  chunk's hidden select-all is a no-op.
- Taps and semantics stop at a chunk's first row, so the line above it no
  longer takes taps meant for the row above.
- Column widths keep each table's last measurement: chunks no longer
  built keep sizing their columns until their rows change, instead of all
  widths starting over on every rebuild (e.g. a row selection).
- Cell content of built rows rebuilds only itself; only content of rows
  not built rebuilds the table.
- Add DataTable.build_rows_on_demand (default True), also honored by
  DataTable2, to build every row.
- DataTable2: look the theme up once, clamp the column count at 0, fix
  the docstring's border condition.
The chunk key only tracked control ids and plain text, so a cell whose
content changed without changing either - a Container resized out of
view, or text edited while shown and restored after scrolling away -
kept its old width until the chunk was built again.

Control now counts its notify() calls (changeCount), and the chunk key
includes the cell's and its content's count, so any change since the
last table build drops that chunk's measurement.
- Snapshot a chunk's key when it stops being built, and compare later
  table builds against that, not against the previous table build: a
  built chunk measures again whenever its cells change size, so an edit
  made while it was shown no longer drops the width it measured.
- Make every control inside the content of a row that isn't built notify
  its parent, so a nested change (a Text inside a Container cell) counts
  as a change of the cell, rebuilds the table and drops that chunk's
  cached width. Building the row turns it off again.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants