Repository navigation
Load and scroll long tables smoothly: lazy DataTable and DataTable2 rows, no compositing layer per control - #7076
Open
FeodorFitsner wants to merge 14 commits into
Open
FeodorFitsner wants to merge 14 commits into
FeodorFitsner wants to merge 14 commits into
Conversation
…pacity Every control was wrapped in Opacity, also at the default 1.0, to keep the tree's shape stable. Flutter's RenderOpacity is a repaint boundary with its own OpacityLayer at any opacity above 0, so each control had a layer. A 3,470-row DataTable (about 35k controls) spent ~6 s of frame work in a 30-step scroll on web, walking those layers every frame. _ControlOpacity keeps the always-present wrapper but paints its child directly at 1.0, nothing at 0.0, and pushes an OpacityLayer only in between. The same scroll now takes 47 ms of frame work. DataTableControl paints the table behind a RepaintBoundary, so a scrolling parent moves the recorded picture instead of repainting every row, and reads horizontal_lines (it read "horizontal_lLines", so the lines never showed). Bump the flet Flutter package to 1.1.3.
Deploying flet-website-v2 with
|
| Latest commit: |
34599fb
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://a68ca1c5.flet-website-v2.pages.dev |
| Branch Preview URL: | https://feature-datatable-repaint.flet-website-v2.pages.dev |
…iewport A DataTable of 100+ rows of one fixed height renders as 10-row DataTable chunks under a header DataTable, building only the chunks near the nearest vertical Scrollable's viewport. Chunks share column widths through a TableColumnWidth that keeps each column's widest measured cell, seeded by an offstage table of each column's widest texts. On a 3,470-row x 9-column table: the load frame drops from 3.4 s to 0.16 s, and selecting a row from ~1.2 s to under 50 ms. Screenshots match one DataTable pixel for pixel.
Unmodified copy of data_table_2's DataTable2 (BSD 3-Clause, LICENSE kept) in place of the pub dependency, so the next commit can change how it builds its rows. The SDK floor rises to 3.8 for the copy's null-aware elements; Flutter 3.47 is already required. typos skips the copy, which keeps upstream's spelling.
A DataTable2 of 100+ rows with its heading row fixed and no outer border builds its data rows in 10-row Tables that a ListView creates as they scroll into view, instead of one Table of every cell; fixed left columns get their own ListView on the synced controller. Each chunk after the first strokes the line between rows above itself, as Table does. The per-row parts of build() move into methods both paths share. On a 3,470-row x 9-column table the load frame drops from 2.5-2.8 s to 0.27 s. Screenshots match upstream DataTable2 pixel for pixel at ten scroll positions; with fixed columns, checkboxes, lines and taller rows, the only differences are 1-21/255 of anti-aliasing at chunk boundaries.
1 task done
The handling_events, sortable_and_selectable and spacing examples set DataTable.horizontal_lines, which had no effect until this PR fixed the "horizontal_lLines" key. Their goldens change only in the row lines; the new images match the CI runner's screenshots pixel for pixel.
The Debian 10 build failed when an archive mirror served a truncated package and apt then hung until the 15-minute step timeout. In the Debian 10 container apt now gives up on a stalled download after 30 s and retries failed files (Acquire::Retries 5), so it moves on to the next mirror; apt-get update and install are retried up to three times in every Linux container.
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
The lazy DataTable implementation can crash on zero-height rows, retain stale or inconsistent column widths, and expose invisible checkboxes with misleading actions to screen readers.
4 open issues
What changed in this PR
Improves long-table loading and scrolling in Flet by building rows near the viewport and removing unnecessary opacity layers, while preserving the stable control wrapper introduced in #6928.
Changes:
- Adds lazy rendering for eligible
DataTableandDataTable2configurations, with shared column sizing and synchronized fixed columns. - Optimizes opacity painting and table repaints, and fixes
DataTable.horizontal_lines. - Adds widget tests and documentation, bumps the Flutter package to 1.1.3, and adds Linux dependency-install retries.
| File | Description |
|---|---|
sdk/python/packages/flet/src/flet/controls/material/datatable.py |
Documents lazy rendering requirements and column sizing. |
sdk/python/packages/flet-datatable2/src/flutter/flet_datatable2/test/datatable2_test.dart |
Tests lazy rows, scrolling, fixed columns, and eager fallback. |
sdk/python/packages/flet-datatable2/src/flutter/flet_datatable2/pubspec.yaml |
Removes the upstream dependency and updates SDK/test requirements. |
sdk/python/packages/flet-datatable2/src/flutter/flet_datatable2/lib/src/utils/datatable.dart |
Uses vendored column-size definitions. |
sdk/python/packages/flet-datatable2/src/flutter/flet_datatable2/lib/src/datatable2.dart |
Switches the control to the vendored implementation. |
sdk/python/packages/flet-datatable2/src/flutter/flet_datatable2/lib/src/data_table_2/LICENSE |
Retains the upstream BSD license. |
sdk/python/packages/flet-datatable2/src/flutter/flet_datatable2/lib/src/data_table_2/data_table_2.dart |
Adds vendored table rendering with lazy row chunks. |
sdk/python/packages/flet-datatable2/src/flutter/flet_datatable2/lib/src/data_table_2/data_table_2_resizable.dart |
Includes upstream column-resizing support. |
sdk/python/packages/flet-datatable2/src/flet_datatable2/datatable2.py |
Documents DataTable2 lazy-rendering requirements. |
sdk/python/.pre-commit-config.yaml |
Excludes vendored source from spelling checks. |
packages/flet/test/widgets/datatable_test.dart |
Tests rendering, borders, sizing, scrolling, and selection. |
packages/flet/test/widgets/control_opacity_test.dart |
Tests opacity layers and child-state preservation. |
packages/flet/pubspec.yaml |
Bumps the Flutter package to 1.1.3. |
packages/flet/lib/src/controls/datatable.dart |
Adds lazy chunks, shared widths, repaint boundaries, and horizontal-line parsing. |
packages/flet/lib/src/controls/base_controls.dart |
Avoids opacity layers at fully opaque and transparent values. |
packages/flet/CHANGELOG.md |
Records Flutter rendering improvements and fixes. |
client/pubspec.lock |
Updates Flet and removes the upstream table dependency. |
CHANGELOG.md |
Records user-facing performance improvements and the line fix. |
.github/workflows/ci.yml |
Adds apt retries and Debian archive-download timeouts. |
Review effort: Balanced
…t-out - Follow the ScrollPosition itself, so a position a Scrollable swaps in keeps building rows as it scrolls. - Keep the hidden rows of chunk and header tables (heading checkbox, stand-in rows) and the offstage seed table out of keyboard focus; a chunk's hidden select-all is a no-op. - Taps and semantics stop at a chunk's first row, so the line above it no longer takes taps meant for the row above. - Column widths keep each table's last measurement: chunks no longer built keep sizing their columns until their rows change, instead of all widths starting over on every rebuild (e.g. a row selection). - Cell content of built rows rebuilds only itself; only content of rows not built rebuilds the table. - Add DataTable.build_rows_on_demand (default True), also honored by DataTable2, to build every row. - DataTable2: look the theme up once, clamp the column count at 0, fix the docstring's border condition.
The chunk key only tracked control ids and plain text, so a cell whose content changed without changing either - a Container resized out of view, or text edited while shown and restored after scrolling away - kept its old width until the chunk was built again. Control now counts its notify() calls (changeCount), and the chunk key includes the cell's and its content's count, so any change since the last table build drops that chunk's measurement.
- Snapshot a chunk's key when it stops being built, and compare later table builds against that, not against the previous table build: a built chunk measures again whenever its cells change size, so an edit made while it was shown no longer drops the width it measured. - Make every control inside the content of a row that isn't built notify its parent, so a nested change (a Text inside a Container cell) counts as a change of the cell, rebuilds the table and drops that chunk's cached width. Building the row turns it off again.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Description
A web app with a long
DataTable(3,470 rows × 9 columns, about 35k controls) freezes for about 4 s when the table loads, freezes again for about a second whenever a row is selected, and stutters on every scroll. Embedded in Flet Studio's preview, it slows all of Studio. The scrolling stutter comes from how every control is wrapped. The freezes come fromDataTableitself, which builds, measures and lays out every cell.Cause
Since #6928,
LayoutControlandBaseControlwrap every control inOpacity, including at the default 1.0, so adding or changingopacitydoesn't rebuild the subtree. The comment there saysOpacityat 1.0 paints its child without a layer. That's no longer true: Flutter'sRenderOpacityis a repaint boundary (isRepaintBoundary => alwaysNeedsCompositing, which ischild != null && alpha > 0). It has its ownOpacityLayerat any opacity above 0, 1.0 included.So every control was its own layer with its own picture. On web, each frame walks the whole layer tree (
measureChildren,visitPicture,Picture.cullRect), off-screen layers included. In a CPU profile of a 30-step wheel scroll, that walk was about two-thirds of all main-thread time.Fix
_ControlOpacityreplacesOpacityin_opacity(). The wrapper is still always there, so the tree keeps its shape, but it paints the child directly at 1.0, paints nothing at 0.0, and pushes anOpacityLayeronly in between. This is howRenderOpacityworked before it became a repaint boundary.animate_opacitystill usesAnimatedOpacity.DataTableControlpaints the table behind aRepaintBoundary.RenderTable.paintpaints every row with no culling, so inside a scrolling parent each frame re-recorded the whole table. With the boundary, a scroll moves the recorded picture instead. In a plain Flutter repro (no Flet), this alone cut scroll frames over 50 ms from 40 to 0.DataTablebuilds only the rows near the viewport when it has 100 or more rows of one height (the default) and its rows are all selectable or none. The rows go into FlutterDataTables of 10 rows each, with their heading row clipped off, stacked under a headerDataTable. Only the chunks near the nearest verticalScrollable's viewport are built, so the table still scrolls with its parentColumn, as before; no API change. Rows, checkboxes, ink, sort arrows and lines are still drawn by Flutter'sDataTable.TableColumnWidththat keeps each column's widest cell measured so far. An offstage table seeds them with each column's widest texts, picked by adding up per-character advances, so a long value far down the table sizes its column before it scrolls into view. Widths are measured afresh when the rows are replaced.select_changeto every row, as before.data_row_max_height=float("inf")), and tables that mix selectable and unselectable rows still render as oneDataTable.DataTable2(flet-datatable2) builds only the rows near the viewport when it has 100 or more rows, its heading row fixed (the default) and no outer border.data_table_2builds every row into oneTable, so a 3,470-row table froze the app for about 2.5 s on load.flet-datatable2now carries its own copy ofdata_table_23.0.0'sDataTable2(BSD 3-Clause, LICENSE kept), committed unmodified first so the second commit shows only the change:Tables that aListViewbuilds as they scroll into view, with exact row heights (specific_row_heightincluded). Fixed left columns get their ownListViewon the existing synced controller. Each chunk after the first strokes the line between rows above itself, centered on the boundary asTabledoes.build()(row decoration, data cell, row checkbox) became methods that both paths use, so the eager path is unchanged.DataTableControlreadshorizontal_lines. It read"horizontal_lLines", soDataTable.horizontal_linesnever had any effect.fletFlutter package is bumped to 1.1.3.handling_events,sortable_and_selectableandspacingDataTable examples are updated. Those examples sethorizontal_lines, which now draws; the images differ only in the row lines, and match the CI runner's screenshots pixel for pixel.apt-get update/installup to three times. In the Debian 10 container, apt also gives up on a stalled download after 30 s and retries failed files. Previously an archive mirror serving a truncated package hung the job until its 15-minute timeout.Measurements
Loading the 3,470-row app (CPU profile, profile build):
The same app in Flet Studio's preview: the longest frame while the table loads drops from 3,867 ms to 217 ms. Selecting a row in a 3,470-row table with a checkbox column: frames over 50 ms per click drop from 3 (one of 950–1,333 ms) to none.
Screenshots of the 3,470-row app match
mainpixel for pixel at 1400 and 1920 px wide and at ten scroll positions down to the last row. A 500-row table with a checkbox column, sorting, grid lines, a rounded border, edit icons and hover color matches through hovering, selecting, sorting and scrolling, apart from a handful of anti-aliased pixels where lines cross (at most 12/255 off).The same 3,470 rows in a
DataTable2: the longest frame drops from 2,517–2,833 ms to 267 ms (75 ms of frame work). Screenshots match upstreamDataTable2pixel for pixel at ten scroll positions, at 1920 and 1000 px wide. A 500-rowDataTable2with a checkbox column, two fixed left columns, grid lines, sorting, hover color,bottom_marginand taller rows matches through hovering, selecting, sorting, select-all and both scroll directions, apart from 1–21/255 of anti-aliasing along chunk boundaries.Scrolling:
Measured on the 3,470-row app in headless Chrome 157, scrolled with the mouse wheel.
CPU profile of a 30-step scroll, profile builds:
RepaintBoundaryonlyFrames longer than 50 ms during a 40-step scroll, release builds:
Before the lazy table, most of what was left was hit testing:
RenderTable.hitTestChildrenchecks every cell on each pointer event, about 1 s over the 30 steps. The lazy table only hit-tests the rows it built; with it, the same 40-step scroll has no frames over 50 ms either (longest 33–50 ms).Test code
3,000 rows in a scrolling
Column, plus a box with buttons that set its opacity to 1.0, 0.4 and 0.0. Run it withflet run --webon 1.1.2 and on this branch, then scroll with the mouse wheel or drag the scrollbar. On 1.1.2 the page freezes for a few seconds while the table appears, and then stutters as it scrolls; on this branch the table appears at once and scrolls smoothly. The opacity buttons should show the box solid, faded and hidden, and the table should have horizontal lines (missing on 1.1.2).With this app, 40 wheel steps in headless Chrome gave 45–46 frames over 50 ms on 1.1.2 and none on this branch.
For
DataTable2, the same rows in aDataTable2(pip install flet-datatable2). On 1.1.2 the page freezes for a couple of seconds before the table shows; on this branch it shows at once. The table scrolls itself, with its header fixed:Type of change
Checklist
website/sidebars.ymlfor breaking changes, removals, and deprecations, if applicable.Additional details
test/widgets/control_opacity_test.dartchecks for no layer at 1.0 or 0.0, anOpacityLayerat 0.5, and that the child's state survives opacity changes.test/widgets/datatable_test.dartchecks theRepaintBoundary, thathorizontal_lines/vertical_linesreach theTableBorder, and for a long table: only rows near the viewport are built and sit where one table would put them; columns line up and fit a long text (plain orTextcontrol) far down the table; the header checkbox selects rows not yet built and shows rows selected out of view. The fullfletpackage suite passes (191 tests).flet_datatable2/test/datatable2_test.dart(new) checks that a longDataTable2builds only rows near the viewport, reaches its last row, keeps fixed left columns in line while scrolling, and that a short one is still oneTable.DataTableandDataTable2docstrings say when a table builds only its visible rows, and what that needs.