Newb Question – Mixing a regular UI with low level rendering

Help
1

Hi

Sorry for the long post. I am new to Flutter and Dart, so I am trying to give a bit of context so people can understand the ravings of a newbie!

I am building a (non game) app (for all platforms) to render S-57 navigation charts. The app has two distinct parts:

  • a regular UI with dialogs, buttons, text fields, blah blah
  • a low level renderer to display the charts (the charts are not tile based, but use polygon, line, and point geometries)

The chart renderer needs to be:

  • fast (ish?) – I have to render a lot of polygons (I can triangulate them), and as users navigate the map, re-compute new polygons and re-render them
  • embedded within the app as some sort of drawable surface widget (so not in a separate window – think of a Canvas node in Swing/JavaFx/HTML)

I am not sure how the embedding of the chart renderer part will work in Flutter.

I have done this kind of thing in other UI toolkits before, doing one of two things:

  • using a “canvas-like” widget from the toolkit, which usually doesn’t give great performance because the “canvas” is too far abstracted from the GPU (e.g. usually using (semi) immediate mode rendering, no shaders, etc)
  • hacking together two UI rendering loops, one for the UI part and one for the low level renderer (usually OpenGL), and this was ugly-as and fragile.

[I should have added here that I do not necessarily need an OpenGL style game loop. Happy to redraw my map when Flutter decides the UI needs redrawing in response to events]

Reading the Flutter docs I get the impression that things might be different in Flutter, and that combining the two parts of my app might be relatively easy in Flutter.

So I have two direct UI questions:

  • is https://api.flutter.dev/flutter/dart-ui/Canvas-class.html the Flutter Canvas widget (if it’s fast enough then it will do fine)?
  • is there some kind of lower level drawing surface that I can use in a widget in my app? It looks like there is since I see things like FragmentShader classes, but I am not sure where to start …

Finally one other question about power consumption on mobile. If the Flutter engine/run-time is re-rendering the scene every “OpenGL-ish tick”, then on mobile this is going to kill the battery. But I am guessing it doesn’t do this?

thanks and again sorry for the giant post.
CW

2

In this case, you can use a CustomPaint. Canvas is not a Widget (I know, but, trust me, not everything is a Widget in Flutter - they lied to us! :angry:)

(video included)

If you really need to go down the rabbit role, you can always use shaders: https://www.youtube.com/watch?v=OpcPZdfJbq8&t=838s

But I think CustomPaint covers your intention.

3

CustomPaint can access a Canvas object to draw on.

That Canvas looks like a traditional GraphicsContext, but with some lower level stuff like drawing collections of vertices, which is very nice, and may indeed be all I need (have to do some experiments! :slight_smile: )

Can I get any lower than that – access to things like vertex buffers, triangle strips etc?

thanks
cw

4

This: engine/docs/impeller/Flutter-GPU.md at main · flutter/engine · GitHub

Flutter GPU enables you to build arbitrary renderers from scratch using just Dart and GLSL. No native platform code required.

1 Like
5

Just to add some more info for anyone who might read this. Canvas actually has VertexMode which allows your triangles to be interpreted as Strips, Fans, etc. Very nice!

6

To wrap this thread up – the Flutter canvas is very fast, much faster than other canvas implementations I have used (probably due to having some slightly lower level features than most canvas implementations).

Comfortably rendering millions of triangles per frame.

Kudos to the Flutter team.

2 Likes
7

Awesome to hear!!

8

If I may interject, an alternative to canvas is the three_JS package. Canvas is faster but the three_JS package provides high-level 3D objects like cameras, lighting meshes and scenes. It handles the technical challenges of projections, Raycasting, extrusions, and index generation. It can be easier to use avoiding the direct manipulation of float 32 arrays. However, these differences mostly relate to full 3D worlds and may not have as much impact in a charting app. Just sayin’.