How does the Dart VM manage OS threads under the hood when multiple isolates are created?

Help Dart
1

I understand that each isolate has its own memory and event loop, but I’m curious whether isolates are backed by dedicated OS threads, or if there’s a thread pool or some form of thread reuse mechanism.

3 Likes
2

@mraleph it would be aw if you could shed some light on this.
As isolates use available CPU cores I expect that they are wrapped in OS threads and probably use a pool of isolates per core or something like that

3

This depends on how embedder manages isolates: in Flutter UI isolates are pinned to a specific UI thread[1], while the rest are using a thread pool owned by the isolate group. The default maximum size of this thread pool is proportional to the size of the new space, because each mutator grabs a chunk of the new space as a thread local allocation buffer (TLAB), see this code - so only this many isolates belonging to the same isolate group can execute in parallel. Though if workers become blocked (e.g. an isolate blocks on a synchronous FFI call) thread pool is allowed to grow to beyond this number of threads to accommodate this.

I understand that each isolate has its own memory

Strictly speaking all isolates within an isolate group share a GC managed heap. Though you can’t observe that from Dart because Dart does not allow sharing mutable objects between isolates.

  1. Flutter is in the process of switching from using a dedicated UI thread, to the platform UI thread. See this issue for more details. ↩︎

7 Likes
4

I’m not an expert by any stretch of imagination but I think it’s a good idea to skim through the code of the Isolate class on the C++ side. I don’t understand much of it but, nevertheless, I find it demystifies the VM a little bit for me, especially once you’re somewhat familiar with C++ threads.

Header here, implementation here:

Whenever you see #if !defined(PRODUCT) … #endif, those lines are only there for non-release builds, IIUC.

@mraleph, I know there’s the experimental @pragma('vm:shared') annotation that relates to your multithreading proposal (here). Is that still alive and well?

2 Likes
5

Yes, we are in the process of implementing this.

3 Likes
6

Thanks!

I want to make sure I understand a comment you made on that linked issue:

First of all, the type of code you have written can already be written without any issues: you can allocate memory for Int64List outside of the Dart heap and then share that between multiple isolates. Then process that concurrently.

To allocate memory for Int64List outside the Dart heap, I need to use FFI, correct? I can’t, for example, use some invocation on Int64List or ByteBuffer, can I?

Because, if I understand correctly, this might be all I need. I have a blob of long-living data that should be updated ~once per frame, and I want to update it from a separate isolate but read it from the UI thread. So you’re saying all I need to do is make a C shared lib with functions like alloc_for_dart() and free_for_dart(), and then somehow wrap that memory in a Dart ByteBuffer?

7

@filip you’re not the first to task this :wink:

I have a memory of @mraleph actually giving me a code sketch on one of the issues about this in the dart lang github repo where you could use the Dart FFI API to do this but without actually needing any native code, you could just do it all from Dart code but I cant for the life of me find that issue or the comment now.

1 Like
8

You need to use FFI but you should not need to write any native code for that:

import 'package:ffi/ffi.dart';

final mem = malloc.allocate<Uint8>(n);
final list = mem.asTypedList(n, finalizer: malloc.nativeFree);

// You can sent `mem` to another isolate and there `asTypeList` it.

This is all you need. Just need to be careful with finalization (don’t attach multiple finalizers or you would need more complicated lifetime management scheme with ref counting).

7 Likes
9

Thanks @mraleph , this time I am going to make sure to bookmark this answer! :slight_smile:

10

This is fantastic, thanks!

I went ahead and tested it on the quicksort example from the linked issue. And it works!

import 'dart:ffi';
import 'dart:isolate';
import 'dart:math';
import 'dart:typed_data';

import 'package:ffi/ffi.dart';

void main() async {
  const n = 40;
  final pointer = malloc.allocate<Int64>(n * Int64List.bytesPerElement);
  final array = pointer.asTypedList(n, finalizer: malloc.nativeFree);

  final random = Random();
  for (var i = 0; i < n; i++) {
    array[i] = random.nextInt(10);
  }

  print('Original array: $array');

  final receivePort = ReceivePort();
  final isolate = await Isolate.spawn(isolateEntry, [
    receivePort.sendPort,
    n,
    pointer,
  ]);

  bool working = true;
  receivePort.listen((message) {
    if (message is String) {
      print('Message: $message');
      receivePort.close();
      isolate.kill();
      working = false;
    }
  });

  while (working) {
    print('Current WIP array: $array');
    await Future<void>.delayed(Duration(milliseconds: 50));
  }

  print('Final: $array');
}

void isolateEntry(List<dynamic> args) async {
  final sendPort = args[0] as SendPort;
  final n = args[1] as int;
  final pointer = args[2] as Pointer<Int64>;
  final array = pointer.asTypedList(n);

  await _quickSort(array, 0, array.length - 1);

  sendPort.send("DONE");
}

int _partition(Int64List array, int low, int high) {
  int pivot = array[high];
  int i = (low - 1);

  for (var j = low; j < high; j++) {
    if (array[j] <= pivot) {
      _swap(array, ++i, j);
    }
  }

  _swap(array, i + 1, high);

  return i + 1;
}

Future<void> _quickSort(Int64List array, int low, int high) async {
  if (low < high) {
    int partitionIndex = _partition(array, low, high);

    // Introduce synthetic delay.
    await Future<void>.delayed(const Duration(milliseconds: 100));

    await Future.wait<void>([
      _quickSort(array, low, partitionIndex - 1),
      _quickSort(array, partitionIndex + 1, high),
    ]);
  }
}

void _swap(Int64List array, int i, int j) {
  final tmp = array[i];
  array[i] = array[j];
  array[j] = tmp;
}
5 Likes