Unverified Commit e828f1e5 authored by Jimmy Miller's avatar Jimmy Miller Committed by GitHub

Add webpack utils (#99826)

Okay, so for these hashes the way it works in webpack is they have an
assembly script implementation that they then go and compile to wasm and
embed that wasma as a base64 encoded string.

I felt introducing wasm was a bit overkill. My first thought was doing
this with rust. But even giving it the best possible chance of winning
(using workers vs processes) it doesn't win on these smaller cases. It
does on the larger ones.

Judging all of this, I thought the easiest answer is to just do some
iterations with an agent on making these as fast as I could in js.

Agents are quite good at algorithms like this. I did do some random
testing of a ton of inputs and everything matches up. The speed reults
are still a mixed bag.

| Workload | JS | WASM | Integrated Rust | Rust vs JS | JS vs WASM |
| ------------------------------ | -------: | -------: | --------------:
| ----------- | ----------- |
| CSS MD4, 100 classes ×2 | 57 µs | 50 µs | 398 µs | 7.0× slower | 1.1×
slower |
| CSS xxhash64, 100 classes ×2 | 86 µs | 48 µs | 439 µs | 5.1× slower |
1.8× slower |
| CSS MD4, 1,000 classes ×2 | 584 µs | 516 µs | 3,572 µs | 6.1× slower |
1.1× slower |
| CSS xxhash64, 1,000 classes ×2 | 792 µs | 459 µs | 2,445 µs | 3.1×
slower | 1.7× slower |
| Babel MD4, 64 KiB | 108 µs | 70 µs | 65 µs | 1.7× faster | 1.5× slower
|
| Babel MD4, 1 MiB | 1,582 µs | 1,090 µs | 974 µs | 1.6× faster | 1.5×
slower |
| Babel xxhash64, 64 KiB | 41 µs | 4.4 µs | 19 µs | 2.2× faster | 9.3×
slower |
| Babel xxhash64, 1 MiB | 642 µs | 68 µs | 249 µs | 2.6× faster | 9.4×
slower |

Wasm is the winner. But I just wasn't sure it was really by enough to go
add that complexity. The js keeps this as simple as we can get.

Considering this hasn't been implemented earlier, I doubt this is super
crucial, but we can always revisit if needed.

Here are two uses of this functionality as described by the clanker

> CSS Modules (css-loader): hashes the file path and local class name to
generate scoped class names. This means many tiny hashes—roughly one per
selector occurrence, sometimes more for longer generated names. It does
not hash the whole stylesheet.
> Babel (babel-loader): with disk caching enabled, hashes the source,
options, and cache identifier to choose a cache filename. Usually one
source-sized hash per file; no hashing when disk caching is disabled.

I am adding this for completeness. But I think many uses of it (css
modules for example) are going to be things we recommend native
replacements for.

<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:

## For Contributors

We encourage you to use AI to assist you in researching, creating, and
reviewing changes. However, you must review and deeply understand the
contributions you are making. For this reason, **pull request
descriptions from external contributors must be written by a human**.

### Improving Documentation

- Run `pnpm prettier-fix` to fix formatting issues before opening the
PR.
- Read the Docs Contribution Guide to ensure your contribution follows
the docs guidelines:
https://nextjs.org/docs/community/contribution-guide

### Fixing a bug

- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md

### Adding a feature

- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs)
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md

### Signed commits

- This repository requires verified commit signatures on protected
branches.
- If this pull request is blocked for unsigned commits, re-sign the
commits and force-push the branch.
- A `Signed-off-by` line in the commit message is not enough.

## For Maintainers

- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change

### What?

### Why?

### How?

Closes NEXT-
Fixes #

-->
parent 7499b034
import type { ReactNode } from 'react'
export default function Root({ children }: { children: ReactNode }) {
return (
<html>
<body>{children}</body>
</html>
)
}
import utils from './input.utils-test'
import styleLoaderSource from './styles.style-test.txt'
export default function Page() {
return (
<>
<pre id="utils">{JSON.stringify(utils)}</pre>
<pre id="style-loader-source">{styleLoaderSource}</pre>
</>
)
}
module.exports = function (source) {
// Export the pitched source so its requests can be checked without executing them.
return `export default ${JSON.stringify(source)}`
}
import type { NextConfig } from 'next'
const utilsLoader = './utils-loader.js'
const styleLoaders = ['./capture-loader.js', 'style-loader']
const nextConfig: NextConfig = {
turbopack: {
rules: {
'*.utils-test.js': { loaders: [utilsLoader] },
'*.style-test.txt': { loaders: styleLoaders, as: '*.js' },
},
},
webpack(config) {
config.module.rules.push(
{ test: /\.utils-test\.js$/, use: [utilsLoader] },
{ test: /\.style-test\.txt$/, use: styleLoaders }
)
return config
},
}
export default nextConfig
declare module '*.style-test.txt' {
const source: string
export default source
}
module.exports = function () {
const { absolutify, contextify, createHash } = this.utils
return `export default ${JSON.stringify({
relative: contextify(this.rootContext, this.resourcePath + '?q'),
absolute:
absolutify(this.rootContext, './app/input.utils-test.js') ===
this.resourcePath,
hash: createHash().update('abc').digest('hex'),
xxhash: createHash('xxhash64').update('abc').digest('hex'),
webpack: this.webpack === true,
compiler: this._compiler !== undefined,
mode: this.mode,
})}`
}
import { nextTestSetup } from 'e2e-utils'
describe('webpack-loader-utils', () => {
const { next, isNextDev } = nextTestSetup({
files: __dirname,
dependencies: {
'style-loader': '4.0.0',
},
})
it('exposes path and hash utilities to loaders', async () => {
const $ = await next.render$('/')
expect(JSON.parse($('#utils').text())).toEqual({
relative: './app/input.utils-test.js?q',
absolute: true,
hash: '44bc2cf5ad770999',
xxhash: '44bc2cf5ad770999',
webpack: !process.env.IS_TURBOPACK_TEST,
compiler: !process.env.IS_TURBOPACK_TEST,
mode: isNextDev ? 'development' : 'production',
})
})
it('runs style-loader pitching with the real loader context', async () => {
const $ = await next.render$('/')
const source = $('#style-loader-source').text()
expect(source).toContain('injectStylesIntoStyleTag.js')
expect(source).toContain('!!./styles.style-test.txt')
})
})
This diff is collapsed.
...@@ -19,6 +19,7 @@ import { ...@@ -19,6 +19,7 @@ import {
import fs from 'fs' import fs from 'fs'
import path from 'path' import path from 'path'
import Module, { createRequire } from 'module' import Module, { createRequire } from 'module'
import { absolutify, contextify, createHash } from './webpack-loaders-utils'
export type { IpcInfoMessage, IpcRequestMessage } from './transforms' export type { IpcInfoMessage, IpcRequestMessage } from './transforms'
...@@ -251,6 +252,7 @@ const transform = async ( ...@@ -251,6 +252,7 @@ const transform = async (
target, target,
mode, mode,
sourceMap, sourceMap,
utils: { absolutify, contextify, createHash },
getOptions() { getOptions() {
const entry = this.loaders[this.loaderIndex] const entry = this.loaders[this.loaderIndex]
return entry.options && typeof entry.options === 'object' return entry.options && typeof entry.options === 'object'
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment