Optimize next-app-loader file resolution (#51924)
This PR changes `fs.stat` calls (to check if a file exists or not) to be
aggregated as `fs.readdir` calls and cached during the entire loader
pass. This is because we have a huge amount of file to check, but they
are always in a small amount of directories.
For example, a route that's 5-directory deep (`/a/b/c/d/page.js`) can
have up to 4,000 special segments and metadata files to check. However
they're always under these 5 directories. So instead of 4,000 `fs.stat`
calls let's just do 5 `fs.readdir` calls.
Didn't measure it carefully but a quick local test shows a 20~30%
improvement.
<details>
<summary>Another idea for future improvements</summary>
Currently, we create a MissingModuleDependency for any combination of
possible metadata filename. But in reality, we only need to track them
incrementally by index. For example if `icon1.js` is missing, it's kinda
waste to still track `icon2.js`~`icon9.js` because `icon1.js` will be
added first. This change should at least reduce the number of file
watchers by 80% and the initial compilation time by 0.5~1s, depending on
the actual route.
</details>
---------
Co-authored-by:
Tim Neutkens <tim@timneutkens.nl>
Showing
Please
register
or
sign in
to comment