Unverified Commit 859de65b authored by vercel-fleet-prod[bot]'s avatar vercel-fleet-prod[bot] Committed by GitHub

configureRayonThreadPool

Co-authored-by: 's avatarNiklas Mischkulnig <4586894+mischnic@users.noreply.github.com>
parent ba6b218b
......@@ -10153,6 +10153,7 @@ dependencies = [
"napi",
"napi-build",
"napi-derive",
"rayon",
"spinning_top",
"talc",
"turbopack-trace-server",
......
......@@ -16,6 +16,7 @@ workspace = true
[dependencies]
napi = { version = "3.10.0", default-features = false, features = ["napi3"] }
napi-derive = { workspace = true }
rayon = { workspace = true }
turbopack-trace-server = { workspace = true, default-features = false }
[target.'cfg(target_family = "wasm")'.dependencies]
......
......@@ -28,3 +28,7 @@ pnpm --filter @vercel/turbopack-trace-server-wasm build
The package pins matching NAPI-RS and Emnapi build dependencies and configures `wasm32-wasip1-threads` as its target. The generated browser loader imports `@napi-rs/wasm-runtime` and `@emnapi/runtime`; consuming code must serve those runtime packages, generated worker files, and the `.wasm` file together.
WebAssembly threads use `SharedArrayBuffer`, so the viewer must be served in a cross-origin isolated context with appropriate COOP and COEP headers.
The browser entry configures Rayon's global thread pool from `navigator.hardwareConcurrency` before exporting the trace server. This is necessary because WASI Preview 1 does not expose the host processor count through `std::thread::available_parallelism()`. If the browser does not report a valid count, the pool defaults to four threads.
Consumers of a generated loader directly must call `configureRayonThreadPool(threadCount)` before handling any viewer messages. The configuration is process-wide and can only be set once; repeated calls with the same count are allowed. Node consumers may use `os.availableParallelism()` or the `RAYON_NUM_THREADS` environment variable to choose the count.
const { readFileSync } = require('node:fs')
const { availableParallelism } = require('node:os')
const {
configureRayonThreadPool,
TurbopackTraceServer,
} = require('./js/turbopack-trace-server-wasm.wasi.cjs')
......@@ -11,6 +13,11 @@ if (!tracePath) {
}
const trace = readFileSync(tracePath)
const requestedThreadCount = Number.parseInt(process.env.RAYON_NUM_THREADS, 10)
configureRayonThreadPool(
requestedThreadCount > 0 ? requestedThreadCount : availableParallelism()
)
new TurbopackTraceServer(trace, (progress) => {
const line = progress.done
? `Initial read completed (${Math.floor(
......
use std::{
ops::ControlFlow,
sync::Mutex,
time::{Duration, Instant},
};
......@@ -21,6 +22,40 @@ static ALLOC: talc::sync::TalcLock<
> = talc::sync::TalcLock::new(talc::wasm::WasmGrowAndClaim);
const PROGRESS_INTERVAL: Duration = Duration::from_millis(250);
static RAYON_THREAD_COUNT: Mutex<Option<usize>> = Mutex::new(None);
/// Configures the global Rayon pool before any trace query initializes it.
#[napi]
pub fn configure_rayon_thread_pool(thread_count: u32) -> Result<u32> {
if thread_count == 0 {
return Err(napi::Error::from_reason(
"Rayon thread count must be greater than zero",
));
}
let thread_count = thread_count as usize;
let mut configured_thread_count = RAYON_THREAD_COUNT
.lock()
.map_err(|_| napi::Error::from_reason("Rayon thread pool configuration lock poisoned"))?;
if let Some(configured_thread_count) = *configured_thread_count {
if configured_thread_count == thread_count {
return Ok(thread_count as u32);
}
return Err(napi::Error::from_reason(format!(
"Rayon thread pool is already configured with {configured_thread_count} threads"
)));
}
rayon::ThreadPoolBuilder::new()
.num_threads(thread_count)
.build_global()
.map_err(|error| {
napi::Error::from_reason(format!("failed to configure Rayon thread pool: {error}"))
})?;
*configured_thread_count = Some(thread_count);
Ok(thread_count as u32)
}
#[napi(object)]
pub struct TraceLoadProgress {
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment