Performance optimization when working with Sockets

[sagarsuri56]

Currently I am working on a project which deals with bi-directional Socket connection. Since there is a lot of JSON data coming from Socket and we need to make sure that CPU and memory usage doesn’t really go high making it hard to have a smooth experience.

I am curious to know if someone have experience working with Socket and dealing with a lot of data. What optimization techniques you used for performance?

[escamoteur]

First of all, why do you think you have a problem. Optimization without benchmarking is almost never a good idea.
I recommend reading HttpClients: Essential Facts and Tips

For how to optimize your Httpclients

[filip]

@sagarsuri56 I’d be interested to see your measurements. If you’re seeing high CPU utilization, then what does DevTools CPU profile tell you? Is the CPU really spending that much time dealing with sockets?

One hypothesis would be that the vast majority of the work is actually spent encoding and decoding from JSON. If that’s the case, then optimizing sockets would be wasting your time. Amdahl’s law. Optimizing parsing is a whole different problem from optimizing networking sockets.

[Sbrobow]

Also, you could optimize bandwidth by not using JSON at all. You could use MessagePack, ProtoBuf, etc., since sockets are binary (AFAIK, only the first version of WebSockets was text-only, but I’m not sure). Or you could compress your JSON using something like LZString (all those have packages in pub.dev).

If serialization/deserialization is an issue, you should maybe delegate your network stack to an isolate using integral_isolates (but, then again, isolate communication has its costs, since Dart doesn’t share memory, so it needs to copy it).

And, without using Isolates, I really doubt Dart can saturate a CPU (it’s single thread, it cannot make a CPU go vruuuum, unless you use isolates).

[escamoteur]

Maybe we let him first tell us what problems he has observed?

[sagarsuri56]

After doing analysis regarding performance. Here is what I came up with:

Here is how the app looks like. 4 out of 5 bottom tabs show a list of live price of products from socket. The top tabs also show list of products based on certain criteria. So we make one HTTP call to get list of products. Then open socket connections to fetch live prices of these products.

JSON is the format for transmitting and consuming data. @Sbrobow @filip
Even I have a strong suspicion that one of the reason for high energy impact is JSON encoding and decoding.

This is the socket library we are working with signalr_netcore

The moment we land on this page. The energy impact goes really high.
CPU usage fluctuate between 30-40%. This is how the graph looks like after 2mins of app usage.

We did some testing and figured out that when we shut down all the socket connections. The energy impact becomes negligible but CPU usage was still high.

After further investigation, we figured out that the way we are working with tabs is causing high CPU usage.

[escamoteur]

So you say even without any network activity you CPU load is high? Or does do you somehow trigger too many rebuilds based on the data you receive?
Without some code or acc6to a sample repository difficult to help I fear.

[filip]

Hey @sagarsuri56, please run the DevTools CPU profiler on your app to learn where the app is spending all that CPU time.

Use the CPU profiler view

If you’re unsure what you’re looking at, the page above explains things well, I think. If you want to share your findings here, please do so. I, for one, would be interested to learn what’s going on.

[Sbrobow]

How this is bad?

According to your graph, you are barely using CPU (the purple blocks).

Of course network is high. You are using it! It was meant to be used!

GPU and Display has nothing to do with network communication (maybe you are overdrawing - that would have a way huger impact on battery and heat than using network). Perhaps try to throttle redraw to 15, 30 FPS? Nobody needs stock price updating 200 FPS >.<

And overhead is???

For me, this is premature optimization.

Of course a device being used will not have energy efficiency.

[sagarsuri56]

Thank you for the link. This is how my CPU profile look like. Note, I have sorted it by “Total Time”:

And this is my the “Self Time”:

You can also see the total UI Jank I have detected just by staying on the page.

[sagarsuri56]

There is just a lot of data coming from socket. As it’s a price data of a product. Hence a lot of rebuilds of ListTile and ListView.builder.

[sagarsuri56]

Since the data is already a lot of data coming from socket. ListView.builder and ListTile is getting rebuilt. I also confirmed using repaint boundary. I think we really need to throttle or else there is no way we can achieve efficiency.

[filip]

Thanks for sharing. It’s hard to draw any conclusions from the screenshot without knowing how long the CPU profiling went and without being able to drill down but I think you’re on the right track here. If the data seems worrying to you, you might want to throttle updates depending on how realtime your app needs to be (e.g. 10 updates per second? 10 updates per minute?).

Good luck!

[sagarsuri56]

Thanks filip! Yeah we are mostly moving towards throttling the incoming data. As it is causing a lot of redrawing of List widgets. Thanks for support. I will definitely share the insights once we have a final conclusion on this