Unverified Commit 0de109ba authored by Javi Velasco's avatar Javi Velasco Committed by GitHub

Refactor Page Paths utils and Middleware Plugin (#36576)

This PR brings some significant refactoring in preparation for upcoming middleware changes. Each commit can be reviewed independently, here is a summary of what each one does and the reasoning behind it:
- [Move pagesDir to next-dev-server](https://github.com/javivelasco/next.js/pull/12/commits/f2fe154c007379f71c14960ddc553eaaaf786ffa) simply moves the `pagesDir` property to the dev server which is the only place where it is needed. Having it for every server is misleading.
- [Move (de)normalize page path utils to a file page-path-utils.ts](https://github.com/javivelasco/next.js/pull/12/commits/27cedf087187b9632ef82a34b3af9cc4fe05d98b) Moves the functions to normalize and denormalize page paths to a single file that is intended to hold every utility function that transforms page paths. Since those are complementary it makes sense to have them together. I also added explanatory comments on why they are not idempotent and examples for input -> output that I find very useful.
- [Extract removePagePathTail](https://github.com/javivelasco/next.js/pull/12/commits/6b121332aa9d3e50bd0f28b691fb7faea1b95f51) This extracts a function to remove the tail on a page path (absolute or relative). I'm sure there will be other contexts where we can use it.
- [Extract getPagePaths and refactor findPageFile](https://github.com/javivelasco/next.js/pull/12/commits/cf2c7b842eebd8c02f23e79345681a794516b646) This extracts a function `getPagePaths` that is used to generate an array of paths to inspect when looking for a page file from `findPageFile`. Then it refactors such function to use it parallelizing lookups. This will allow us to print every path we look at when looking for a file which can be useful for debugging. It also adds a `flatten` helper. 
- [Refactor onDemandEntryHandler](https://github.com/javivelasco/next.js/pull/12/commits/4be685c37e3d1b797e929ea4f31495ed7b00e1cc) I've found this one quite difficult to understand so it is refactored to use some of the previously mentioned functions and make it easier to read.
- [Extract absolutePagePath util](https://github.com/javivelasco/next.js/pull/12/commits/3bc078347426c73491a076d54ef4de977d9da073) Extracts yet another util from the `next-dev-server` that transforms an absolute path into a page name. Of course it adds comments, parameters and examples.
- [Refactor MiddlewarePlugin](https://github.com/javivelasco/next.js/pull/12/commits/c595a2cc629b358cc61861a8a4848b7890d0a15b) This is the most significant change. The logic here was very hard to understand so it is totally redistributed with comments. This also removes a global variable `ssrEntries` that was deprecated in favour of module metadata added to Webpack from loaders keeping less dependencies. It also adds types and makes a clear distinction between phases where we statically analyze the code, find metadata and generate the manifest file cc @shuding @huozhi 

EDIT: 
- [Split page path utils](https://github.com/vercel/next.js/pull/36576/commits/158fb002d02887d7ce4be6747cf550a825a426eb) After seeing one of the utils was being used by the client while it was defined originally in the server, with this PR we are splitting the util into multiple files and moving it to `shared/lib` in order to make explicit that those can be also imported from client. 
parent 7998b63a
...@@ -15,12 +15,11 @@ import { EDGE_RUNTIME_WEBPACK } from '../shared/lib/constants' ...@@ -15,12 +15,11 @@ import { EDGE_RUNTIME_WEBPACK } from '../shared/lib/constants'
import { MIDDLEWARE_ROUTE } from '../lib/constants' import { MIDDLEWARE_ROUTE } from '../lib/constants'
import { __ApiPreviewProps } from '../server/api-utils' import { __ApiPreviewProps } from '../server/api-utils'
import { isTargetLikeServerless } from '../server/utils' import { isTargetLikeServerless } from '../server/utils'
import { normalizePagePath } from '../server/normalize-page-path'
import { normalizePathSep } from '../server/denormalize-page-path'
import { ssrEntries } from './webpack/plugins/middleware-plugin'
import { warn } from './output/log' import { warn } from './output/log'
import { parse } from '../build/swc' import { parse } from '../build/swc'
import { isFlightPage, withoutRSCExtensions } from './utils' import { isFlightPage, withoutRSCExtensions } from './utils'
import { normalizePathSep } from '../shared/lib/page-path/normalize-path-sep'
import { normalizePagePath } from '../shared/lib/page-path/normalize-page-path'
type ObjectValue<T> = T extends { [key: string]: infer V } ? V : never type ObjectValue<T> = T extends { [key: string]: infer V } ? V : never
...@@ -233,7 +232,6 @@ export function getEdgeServerEntry(opts: { ...@@ -233,7 +232,6 @@ export function getEdgeServerEntry(opts: {
isDev: boolean isDev: boolean
page: string page: string
pages: { [page: string]: string } pages: { [page: string]: string }
ssrEntries: Map<string, { requireFlightManifest: boolean }>
}): ObjectValue<webpack5.EntryObject> { }): ObjectValue<webpack5.EntryObject> {
if (opts.page.match(MIDDLEWARE_ROUTE)) { if (opts.page.match(MIDDLEWARE_ROUTE)) {
const loaderParams: MiddlewareLoaderOptions = { const loaderParams: MiddlewareLoaderOptions = {
...@@ -258,10 +256,6 @@ export function getEdgeServerEntry(opts: { ...@@ -258,10 +256,6 @@ export function getEdgeServerEntry(opts: {
stringifiedConfig: JSON.stringify(opts.config), stringifiedConfig: JSON.stringify(opts.config),
} }
ssrEntries.set(opts.bundlePath, {
requireFlightManifest: isFlightPage(opts.config, opts.absolutePagePath),
})
return `next-middleware-ssr-loader?${stringify(loaderParams)}!` return `next-middleware-ssr-loader?${stringify(loaderParams)}!`
} }
...@@ -375,7 +369,6 @@ export async function createEntrypoints(params: CreateEntrypointsParams) { ...@@ -375,7 +369,6 @@ export async function createEntrypoints(params: CreateEntrypointsParams) {
bundlePath: clientBundlePath, bundlePath: clientBundlePath,
isDev: false, isDev: false,
page, page,
ssrEntries,
}) })
}, },
}) })
......
...@@ -63,7 +63,7 @@ import { __ApiPreviewProps } from '../server/api-utils' ...@@ -63,7 +63,7 @@ import { __ApiPreviewProps } from '../server/api-utils'
import loadConfig from '../server/config' import loadConfig from '../server/config'
import { isTargetLikeServerless } from '../server/utils' import { isTargetLikeServerless } from '../server/utils'
import { BuildManifest } from '../server/get-page-files' import { BuildManifest } from '../server/get-page-files'
import { normalizePagePath } from '../server/normalize-page-path' import { normalizePagePath } from '../shared/lib/page-path/normalize-page-path'
import { getPagePath } from '../server/require' import { getPagePath } from '../server/require'
import * as ciEnvironment from '../telemetry/ci-info' import * as ciEnvironment from '../telemetry/ci-info'
import { import {
......
...@@ -26,10 +26,6 @@ import { isDynamicRoute } from '../shared/lib/router/utils/is-dynamic' ...@@ -26,10 +26,6 @@ import { isDynamicRoute } from '../shared/lib/router/utils/is-dynamic'
import escapePathDelimiters from '../shared/lib/router/utils/escape-path-delimiters' import escapePathDelimiters from '../shared/lib/router/utils/escape-path-delimiters'
import { findPageFile } from '../server/lib/find-page-file' import { findPageFile } from '../server/lib/find-page-file'
import { GetStaticPaths, PageConfig } from 'next/types' import { GetStaticPaths, PageConfig } from 'next/types'
import {
denormalizePagePath,
normalizePagePath,
} from '../server/normalize-page-path'
import { BuildManifest } from '../server/get-page-files' import { BuildManifest } from '../server/get-page-files'
import { removePathTrailingSlash } from '../client/normalize-trailing-slash' import { removePathTrailingSlash } from '../client/normalize-trailing-slash'
import { UnwrapPromise } from '../lib/coalesced-function' import { UnwrapPromise } from '../lib/coalesced-function'
...@@ -42,6 +38,8 @@ import isError from '../lib/is-error' ...@@ -42,6 +38,8 @@ import isError from '../lib/is-error'
import { recursiveDelete } from '../lib/recursive-delete' import { recursiveDelete } from '../lib/recursive-delete'
import { Sema } from 'next/dist/compiled/async-sema' import { Sema } from 'next/dist/compiled/async-sema'
import { MiddlewareManifest } from './webpack/plugins/middleware-plugin' import { MiddlewareManifest } from './webpack/plugins/middleware-plugin'
import { denormalizePagePath } from '../shared/lib/page-path/denormalize-page-path'
import { normalizePagePath } from '../shared/lib/page-path/normalize-page-path'
const { builtinModules } = require('module') const { builtinModules } = require('module')
const RESERVED_PAGE = /^\/(_app|_error|_document|api(\/|$))/ const RESERVED_PAGE = /^\/(_app|_error|_document|api(\/|$))/
......
import { webpack5 } from 'next/dist/compiled/webpack/webpack'
/**
* A getter for module build info that casts to the type it should have.
* We also expose here types to make easier to use it.
*/
export function getModuleBuildInfo(webpackModule: webpack5.Module) {
return webpackModule.buildInfo as {
nextEdgeMiddleware?: EdgeMiddlewareMeta
nextEdgeSSR?: EdgeSSRMeta
nextUsedEnvVars?: Set<string>
nextWasmMiddlewareBinding?: WasmBinding
usingIndirectEval?: boolean | Set<string>
}
}
export interface EdgeMiddlewareMeta {
page: string
}
export interface EdgeSSRMeta {
isServerComponent: boolean
page: string
}
export interface WasmBinding {
filePath: string
name: string
}
import { getModuleBuildInfo } from './get-module-build-info'
import { stringifyRequest } from '../stringify-request' import { stringifyRequest } from '../stringify-request'
export type MiddlewareLoaderOptions = { export type MiddlewareLoaderOptions = {
...@@ -8,6 +9,10 @@ export type MiddlewareLoaderOptions = { ...@@ -8,6 +9,10 @@ export type MiddlewareLoaderOptions = {
export default function middlewareLoader(this: any) { export default function middlewareLoader(this: any) {
const { absolutePagePath, page }: MiddlewareLoaderOptions = this.getOptions() const { absolutePagePath, page }: MiddlewareLoaderOptions = this.getOptions()
const stringifiedPagePath = stringifyRequest(this, absolutePagePath) const stringifiedPagePath = stringifyRequest(this, absolutePagePath)
const buildInfo = getModuleBuildInfo(this._module)
buildInfo.nextEdgeMiddleware = {
page: page.replace(/\/_middleware$/, '') || '/',
}
return ` return `
import { adapter } from 'next/dist/server/web/adapter' import { adapter } from 'next/dist/server/web/adapter'
......
import { getModuleBuildInfo } from '../get-module-build-info'
import { stringifyRequest } from '../../stringify-request' import { stringifyRequest } from '../../stringify-request'
export type MiddlewareSSRLoaderQuery = { export type MiddlewareSSRLoaderQuery = {
...@@ -27,7 +28,13 @@ export default async function middlewareSSRLoader(this: any) { ...@@ -27,7 +28,13 @@ export default async function middlewareSSRLoader(this: any) {
absoluteErrorPath, absoluteErrorPath,
isServerComponent, isServerComponent,
stringifiedConfig, stringifiedConfig,
}: MiddlewareSSRLoaderQuery = this.getOptions() } = this.getOptions()
const buildInfo = getModuleBuildInfo(this._module)
buildInfo.nextEdgeSSR = {
isServerComponent: isServerComponent === 'true',
page: page,
}
const stringifiedPagePath = stringifyRequest(this, absolutePagePath) const stringifiedPagePath = stringifyRequest(this, absolutePagePath)
const stringifiedAppPath = stringifyRequest(this, absoluteAppPath) const stringifiedAppPath = stringifyRequest(this, absoluteAppPath)
......
import { getModuleBuildInfo } from './get-module-build-info'
import crypto from 'crypto' import crypto from 'crypto'
export type WasmBinding = {
filePath: string
name: string
}
export default function MiddlewareWasmLoader(this: any, source: Buffer) { export default function MiddlewareWasmLoader(this: any, source: Buffer) {
const name = `wasm_${sha1(source)}` const name = `wasm_${sha1(source)}`
const filePath = `edge-chunks/${name}.wasm` const filePath = `edge-chunks/${name}.wasm`
const binding: WasmBinding = { filePath: `server/${filePath}`, name } const buildInfo = getModuleBuildInfo(this._module)
this._module.buildInfo.nextWasmMiddlewareBinding = binding buildInfo.nextWasmMiddlewareBinding = { filePath: `server/${filePath}`, name }
this.emitFile(`/${filePath}`, source, null) this.emitFile(`/${filePath}`, source, null)
return `module.exports = ${name};` return `module.exports = ${name};`
} }
......
...@@ -6,7 +6,7 @@ import { getUtils, vercelHeader, ServerlessHandlerCtx } from './utils' ...@@ -6,7 +6,7 @@ import { getUtils, vercelHeader, ServerlessHandlerCtx } from './utils'
import { renderToHTML } from '../../../../server/render' import { renderToHTML } from '../../../../server/render'
import { tryGetPreviewData } from '../../../../server/api-utils/node' import { tryGetPreviewData } from '../../../../server/api-utils/node'
import { denormalizePagePath } from '../../../../server/denormalize-page-path' import { denormalizePagePath } from '../../../../shared/lib/page-path/denormalize-page-path'
import { setLazyProp, getCookieParser } from '../../../../server/api-utils' import { setLazyProp, getCookieParser } from '../../../../server/api-utils'
import { getRedirectStatus } from '../../../../lib/load-custom-routes' import { getRedirectStatus } from '../../../../lib/load-custom-routes'
import getRouteNoAssetPath from '../../../../shared/lib/router/utils/get-route-from-asset-path' import getRouteNoAssetPath from '../../../../shared/lib/router/utils/get-route-from-asset-path'
......
...@@ -23,7 +23,7 @@ import { __ApiPreviewProps } from '../../../../server/api-utils' ...@@ -23,7 +23,7 @@ import { __ApiPreviewProps } from '../../../../server/api-utils'
import { acceptLanguage } from '../../../../server/accept-header' import { acceptLanguage } from '../../../../server/accept-header'
import { detectLocaleCookie } from '../../../../shared/lib/i18n/detect-locale-cookie' import { detectLocaleCookie } from '../../../../shared/lib/i18n/detect-locale-cookie'
import { detectDomainLocale } from '../../../../shared/lib/i18n/detect-domain-locale' import { detectDomainLocale } from '../../../../shared/lib/i18n/detect-domain-locale'
import { denormalizePagePath } from '../../../../server/denormalize-page-path' import { denormalizePagePath } from '../../../../shared/lib/page-path/denormalize-page-path'
import cookie from 'next/dist/compiled/cookie' import cookie from 'next/dist/compiled/cookie'
import { TEMPORARY_REDIRECT_STATUS } from '../../../../shared/lib/constants' import { TEMPORARY_REDIRECT_STATUS } from '../../../../shared/lib/constants'
import { addRequestMeta } from '../../../../server/request-meta' import { addRequestMeta } from '../../../../server/request-meta'
......
...@@ -34,10 +34,8 @@ import { NextConfigComplete } from '../server/config-shared' ...@@ -34,10 +34,8 @@ import { NextConfigComplete } from '../server/config-shared'
import { eventCliSession } from '../telemetry/events' import { eventCliSession } from '../telemetry/events'
import { hasNextSupport } from '../telemetry/ci-info' import { hasNextSupport } from '../telemetry/ci-info'
import { Telemetry } from '../telemetry/storage' import { Telemetry } from '../telemetry/storage'
import { import { normalizePagePath } from '../shared/lib/page-path/normalize-page-path'
normalizePagePath, import { denormalizePagePath } from '../shared/lib/page-path/denormalize-page-path'
denormalizePagePath,
} from '../server/normalize-page-path'
import { loadEnvConfig } from '@next/env' import { loadEnvConfig } from '@next/env'
import { PrerenderManifest } from '../build' import { PrerenderManifest } from '../build'
import { PagesManifest } from '../build/webpack/plugins/pages-manifest-plugin' import { PagesManifest } from '../build/webpack/plugins/pages-manifest-plugin'
......
...@@ -13,7 +13,7 @@ import { loadComponents } from '../server/load-components' ...@@ -13,7 +13,7 @@ import { loadComponents } from '../server/load-components'
import { isDynamicRoute } from '../shared/lib/router/utils/is-dynamic' import { isDynamicRoute } from '../shared/lib/router/utils/is-dynamic'
import { getRouteMatcher } from '../shared/lib/router/utils/route-matcher' import { getRouteMatcher } from '../shared/lib/router/utils/route-matcher'
import { getRouteRegex } from '../shared/lib/router/utils/route-regex' import { getRouteRegex } from '../shared/lib/router/utils/route-regex'
import { normalizePagePath } from '../server/normalize-page-path' import { normalizePagePath } from '../shared/lib/page-path/normalize-page-path'
import { SERVER_PROPS_EXPORT_ERROR } from '../lib/constants' import { SERVER_PROPS_EXPORT_ERROR } from '../lib/constants'
import '../server/node-polyfill-fetch' import '../server/node-polyfill-fetch'
import { requireFontManifest } from '../server/require' import { requireFontManifest } from '../server/require'
......
...@@ -50,7 +50,7 @@ import { isBlockedPage, isBot } from './utils' ...@@ -50,7 +50,7 @@ import { isBlockedPage, isBot } from './utils'
import RenderResult from './render-result' import RenderResult from './render-result'
import { removePathTrailingSlash } from '../client/normalize-trailing-slash' import { removePathTrailingSlash } from '../client/normalize-trailing-slash'
import getRouteFromAssetPath from '../shared/lib/router/utils/get-route-from-asset-path' import getRouteFromAssetPath from '../shared/lib/router/utils/get-route-from-asset-path'
import { denormalizePagePath } from './denormalize-page-path' import { denormalizePagePath } from '../shared/lib/page-path/denormalize-page-path'
import { normalizeLocalePath } from '../shared/lib/i18n/normalize-locale-path' import { normalizeLocalePath } from '../shared/lib/i18n/normalize-locale-path'
import * as Log from '../build/output/log' import * as Log from '../build/output/log'
import { detectDomainLocale } from '../shared/lib/i18n/detect-domain-locale' import { detectDomainLocale } from '../shared/lib/i18n/detect-domain-locale'
...@@ -133,7 +133,6 @@ export default abstract class Server<ServerOptions extends Options = Options> { ...@@ -133,7 +133,6 @@ export default abstract class Server<ServerOptions extends Options = Options> {
protected quiet: boolean protected quiet: boolean
protected nextConfig: NextConfigComplete protected nextConfig: NextConfigComplete
protected distDir: string protected distDir: string
protected pagesDir?: string
protected publicDir: string protected publicDir: string
protected hasStaticDir: boolean protected hasStaticDir: boolean
protected pagesManifest?: PagesManifest protected pagesManifest?: PagesManifest
......
import { isDynamicRoute } from '../shared/lib/router/utils'
export function normalizePathSep(path: string): string {
return path.replace(/\\/g, '/')
}
export function denormalizePagePath(page: string) {
page = normalizePathSep(page)
if (page.startsWith('/index/') && !isDynamicRoute(page)) {
page = page.slice(6)
} else if (page === '/index') {
page = '/'
}
return page
}
...@@ -21,14 +21,15 @@ import { BLOCKED_PAGES } from '../../shared/lib/constants' ...@@ -21,14 +21,15 @@ import { BLOCKED_PAGES } from '../../shared/lib/constants'
import { __ApiPreviewProps } from '../api-utils' import { __ApiPreviewProps } from '../api-utils'
import { getPathMatch } from '../../shared/lib/router/utils/path-match' import { getPathMatch } from '../../shared/lib/router/utils/path-match'
import { findPageFile } from '../lib/find-page-file' import { findPageFile } from '../lib/find-page-file'
import onDemandEntryHandler, { import {
entries,
BUILDING, BUILDING,
entries,
onDemandEntryHandler,
} from './on-demand-entry-handler' } from './on-demand-entry-handler'
import { denormalizePagePath, normalizePathSep } from '../normalize-page-path' import { denormalizePagePath } from '../../shared/lib/page-path/denormalize-page-path'
import { normalizePathSep } from '../../shared/lib/page-path/normalize-path-sep'
import getRouteFromEntrypoint from '../get-route-from-entrypoint' import getRouteFromEntrypoint from '../get-route-from-entrypoint'
import { fileExists } from '../../lib/file-exists' import { fileExists } from '../../lib/file-exists'
import { ssrEntries } from '../../build/webpack/plugins/middleware-plugin'
import { difference } from '../../build/utils' import { difference } from '../../build/utils'
import { NextConfigComplete } from '../config-shared' import { NextConfigComplete } from '../config-shared'
import { CustomRoutes } from '../../lib/load-custom-routes' import { CustomRoutes } from '../../lib/load-custom-routes'
...@@ -558,7 +559,6 @@ export default class HotReloader { ...@@ -558,7 +559,6 @@ export default class HotReloader {
isDev: true, isDev: true,
page, page,
pages: this.pagesMapping, pages: this.pagesMapping,
ssrEntries,
}), }),
}) })
} }
...@@ -830,7 +830,9 @@ export default class HotReloader { ...@@ -830,7 +830,9 @@ export default class HotReloader {
) )
}) })
this.onDemandEntries = onDemandEntryHandler(this.watcher, multiCompiler, { this.onDemandEntries = onDemandEntryHandler({
multiCompiler,
watcher: this.watcher,
pagesDir: this.pagesDir, pagesDir: this.pagesDir,
nextConfig: this.config, nextConfig: this.config,
...(this.config.onDemandEntries as { ...(this.config.onDemandEntries as {
......
...@@ -40,7 +40,8 @@ import { ...@@ -40,7 +40,8 @@ import {
isDynamicRoute, isDynamicRoute,
} from '../../shared/lib/router/utils' } from '../../shared/lib/router/utils'
import Server, { WrappedBuildError } from '../next-server' import Server, { WrappedBuildError } from '../next-server'
import { normalizePagePath } from '../normalize-page-path' import { normalizePagePath } from '../../shared/lib/page-path/normalize-page-path'
import { absolutePathToPage } from '../../shared/lib/page-path/absolute-path-to-page'
import Router from '../router' import Router from '../router'
import { getPathMatch } from '../../shared/lib/router/utils/path-match' import { getPathMatch } from '../../shared/lib/router/utils/path-match'
import { hasBasePath, replaceBasePath } from '../router-utils' import { hasBasePath, replaceBasePath } from '../router-utils'
...@@ -94,6 +95,7 @@ export default class DevServer extends Server { ...@@ -94,6 +95,7 @@ export default class DevServer extends Server {
private isCustomServer: boolean private isCustomServer: boolean
protected sortedRoutes?: string[] protected sortedRoutes?: string[]
private addedUpgradeListener = false private addedUpgradeListener = false
private pagesDir: string
protected staticPathsWorker?: { [key: string]: any } & { protected staticPathsWorker?: { [key: string]: any } & {
loadStaticPaths: typeof import('./static-paths-worker').loadStaticPaths loadStaticPaths: typeof import('./static-paths-worker').loadStaticPaths
...@@ -241,10 +243,8 @@ export default class DevServer extends Server { ...@@ -241,10 +243,8 @@ export default class DevServer extends Server {
let resolved = false let resolved = false
return new Promise((resolve, reject) => { return new Promise((resolve, reject) => {
const pagesDir = this.pagesDir
// Watchpack doesn't emit an event for an empty directory // Watchpack doesn't emit an event for an empty directory
fs.readdir(pagesDir!, (_, files) => { fs.readdir(this.pagesDir, (_, files) => {
if (files?.length) { if (files?.length) {
return return
} }
...@@ -256,7 +256,7 @@ export default class DevServer extends Server { ...@@ -256,7 +256,7 @@ export default class DevServer extends Server {
}) })
let wp = (this.webpackWatcher = new Watchpack()) let wp = (this.webpackWatcher = new Watchpack())
wp.watch([], [pagesDir!], 0) wp.watch([], [this.pagesDir], 0)
wp.on('aggregated', async () => { wp.on('aggregated', async () => {
const routedMiddleware = [] const routedMiddleware = []
...@@ -269,20 +269,21 @@ export default class DevServer extends Server { ...@@ -269,20 +269,21 @@ export default class DevServer extends Server {
continue continue
} }
const pageName = absolutePathToPage(
this.pagesDir,
fileName,
this.nextConfig.pageExtensions
)
if (regexMiddleware.test(fileName)) { if (regexMiddleware.test(fileName)) {
routedMiddleware.push( routedMiddleware.push(
`/${relative(pagesDir!, fileName).replace(/\\+/g, '/')}` `/${relative(this.pagesDir, fileName).replace(/\\+/g, '/')}`
.replace(/^\/+/g, '/') .replace(/^\/+/g, '/')
.replace(regexMiddleware, '/') .replace(regexMiddleware, '/')
) )
continue continue
} }
let pageName =
'/' + relative(pagesDir!, fileName).replace(/\\+/g, '/')
pageName = pageName.replace(regexPageExtension, '')
pageName = pageName.replace(/\/index$/, '') || '/'
invalidatePageRuntimeCache(fileName, safeTime) invalidatePageRuntimeCache(fileName, safeTime)
const pageRuntimeConfig = await getPageRuntime( const pageRuntimeConfig = await getPageRuntime(
fileName, fileName,
...@@ -360,12 +361,7 @@ export default class DevServer extends Server { ...@@ -360,12 +361,7 @@ export default class DevServer extends Server {
async prepare(): Promise<void> { async prepare(): Promise<void> {
setGlobal('distDir', this.distDir) setGlobal('distDir', this.distDir)
setGlobal('phase', PHASE_DEVELOPMENT_SERVER) setGlobal('phase', PHASE_DEVELOPMENT_SERVER)
await verifyTypeScriptSetup( await verifyTypeScriptSetup(this.dir, this.pagesDir, false, this.nextConfig)
this.dir,
this.pagesDir!,
false,
this.nextConfig
)
this.customRoutes = await loadCustomRoutes(this.nextConfig) this.customRoutes = await loadCustomRoutes(this.nextConfig)
...@@ -383,7 +379,7 @@ export default class DevServer extends Server { ...@@ -383,7 +379,7 @@ export default class DevServer extends Server {
} }
this.hotReloader = new HotReloader(this.dir, { this.hotReloader = new HotReloader(this.dir, {
pagesDir: this.pagesDir!, pagesDir: this.pagesDir,
distDir: this.distDir, distDir: this.distDir,
config: this.nextConfig, config: this.nextConfig,
previewProps: this.getPreviewProps(), previewProps: this.getPreviewProps(),
...@@ -408,7 +404,7 @@ export default class DevServer extends Server { ...@@ -408,7 +404,7 @@ export default class DevServer extends Server {
eventCliSession(this.distDir, this.nextConfig, { eventCliSession(this.distDir, this.nextConfig, {
webpackVersion: 5, webpackVersion: 5,
cliCommand: 'dev', cliCommand: 'dev',
isSrcDir: relative(this.dir, this.pagesDir!).startsWith('src'), isSrcDir: relative(this.dir, this.pagesDir).startsWith('src'),
hasNowJson: !!(await findUp('now.json', { cwd: this.dir })), hasNowJson: !!(await findUp('now.json', { cwd: this.dir })),
isCustomServer: this.isCustomServer, isCustomServer: this.isCustomServer,
}) })
...@@ -448,7 +444,7 @@ export default class DevServer extends Server { ...@@ -448,7 +444,7 @@ export default class DevServer extends Server {
} }
const pageFile = await findPageFile( const pageFile = await findPageFile(
this.pagesDir!, this.pagesDir,
normalizedPath, normalizedPath,
this.nextConfig.pageExtensions this.nextConfig.pageExtensions
) )
......
import { normalizePagePath, denormalizePagePath } from './normalize-page-path' import { denormalizePagePath } from '../shared/lib/page-path/denormalize-page-path'
import { normalizePagePath } from '../shared/lib/page-path/normalize-page-path'
export type BuildManifest = { export type BuildManifest = {
devFiles: readonly string[] devFiles: readonly string[]
......
...@@ -3,7 +3,7 @@ import type { CacheFs } from '../shared/lib/utils' ...@@ -3,7 +3,7 @@ import type { CacheFs } from '../shared/lib/utils'
import LRUCache from 'next/dist/compiled/lru-cache' import LRUCache from 'next/dist/compiled/lru-cache'
import path from '../shared/lib/isomorphic/path' import path from '../shared/lib/isomorphic/path'
import { PrerenderManifest } from '../build' import { PrerenderManifest } from '../build'
import { normalizePagePath } from './normalize-page-path' import { normalizePagePath } from '../shared/lib/page-path/normalize-page-path'
import { IncrementalCacheValue, IncrementalCacheEntry } from './response-cache' import { IncrementalCacheValue, IncrementalCacheEntry } from './response-cache'
function toRoute(pathname: string): string { function toRoute(pathname: string): string {
......
import { join, sep as pathSeparator, normalize } from 'path'
import chalk from '../../lib/chalk'
import { warn } from '../../build/output/log'
import { promises } from 'fs'
import { denormalizePagePath } from '../normalize-page-path'
import { fileExists } from '../../lib/file-exists' import { fileExists } from '../../lib/file-exists'
import { getPagePaths } from '../../shared/lib/page-path/get-page-paths'
import { nonNullable } from '../../lib/non-nullable'
import { join, sep, normalize } from 'path'
import { promises } from 'fs'
import { warn } from '../../build/output/log'
import chalk from '../../lib/chalk'
async function isTrueCasePagePath(pagePath: string, pagesDir: string) { /**
const pageSegments = normalize(pagePath).split(pathSeparator).filter(Boolean) * Finds a page file with the given parameters. If the page is duplicated with
* multiple extensions it will throw, otherwise it will return the *relative*
const segmentExistsPromises = pageSegments.map(async (segment, i) => { * path to the page file or null if it is not found.
const segmentParentDir = join(pagesDir, ...pageSegments.slice(0, i)) *
const parentDirEntries = await promises.readdir(segmentParentDir) * @param pagesDir Absolute path to the pages folder with trailing `/pages`.
return parentDirEntries.includes(segment) * @param normalizedPagePath The page normalized (it will be denormalized).
}) * @param pageExtensions Array of page extensions.
*/
return (await Promise.all(segmentExistsPromises)).every(Boolean)
}
export async function findPageFile( export async function findPageFile(
rootDir: string, pagesDir: string,
normalizedPagePath: string, normalizedPagePath: string,
pageExtensions: string[] pageExtensions: string[]
): Promise<string | null> { ): Promise<string | null> {
const foundPagePaths: string[] = [] const pagePaths = getPagePaths(normalizedPagePath, pageExtensions)
const [existingPath, ...others] = (
const page = denormalizePagePath(normalizedPagePath) await Promise.all(
pagePaths.map(async (path) =>
for (const extension of pageExtensions) { (await fileExists(join(pagesDir, path))) ? path : null
if (!normalizedPagePath.endsWith('/index')) { )
const relativePagePath = `${page}.${extension}` )
const pagePath = join(rootDir, relativePagePath) ).filter(nonNullable)
if (await fileExists(pagePath)) {
foundPagePaths.push(relativePagePath)
}
}
const relativePagePathWithIndex = join(page, `index.${extension}`)
const pagePathWithIndex = join(rootDir, relativePagePathWithIndex)
if (await fileExists(pagePathWithIndex)) {
foundPagePaths.push(relativePagePathWithIndex)
}
}
if (foundPagePaths.length < 1) { if (!existingPath) {
return null return null
} }
if (!(await isTrueCasePagePath(foundPagePaths[0], rootDir))) { if (!(await isTrueCasePagePath(existingPath, pagesDir))) {
return null return null
} }
if (foundPagePaths.length > 1) { if (others.length > 0) {
warn( warn(
`Duplicate page detected. ${chalk.cyan( `Duplicate page detected. ${chalk.cyan(
join('pages', foundPagePaths[0]) join('pages', existingPath)
)} and ${chalk.cyan( )} and ${chalk.cyan(
join('pages', foundPagePaths[1]) join('pages', others[0])
)} both resolve to ${chalk.cyan(normalizedPagePath)}.` )} both resolve to ${chalk.cyan(normalizedPagePath)}.`
) )
} }
return foundPagePaths[0] return existingPath
}
async function isTrueCasePagePath(pagePath: string, pagesDir: string) {
const pageSegments = normalize(pagePath).split(sep).filter(Boolean)
const segmentExistsPromises = pageSegments.map(async (segment, i) => {
const segmentParentDir = join(pagesDir, ...pageSegments.slice(0, i))
const parentDirEntries = await promises.readdir(segmentParentDir)
return parentDirEntries.includes(segment)
})
return (await Promise.all(segmentExistsPromises)).every(Boolean)
} }
...@@ -55,7 +55,7 @@ import BaseServer, { ...@@ -55,7 +55,7 @@ import BaseServer, {
stringifyQuery, stringifyQuery,
} from './base-server' } from './base-server'
import { getMiddlewareInfo, getPagePath, requireFontManifest } from './require' import { getMiddlewareInfo, getPagePath, requireFontManifest } from './require'
import { normalizePagePath } from './normalize-page-path' import { normalizePagePath } from '../shared/lib/page-path/normalize-page-path'
import { loadComponents } from './load-components' import { loadComponents } from './load-components'
import isError, { getProperError } from '../lib/is-error' import isError, { getProperError } from '../lib/is-error'
import { FontManifest } from './font-utils' import { FontManifest } from './font-utils'
......
import { posix } from '../shared/lib/isomorphic/path'
import { isDynamicRoute } from '../shared/lib/router/utils'
export { normalizePathSep, denormalizePagePath } from './denormalize-page-path'
export function normalizePagePath(page: string): string {
// If the page is `/` we need to append `/index`, otherwise the returned directory root will be bundles instead of pages
if (page === '/') {
page = '/index'
} else if (/^\/index(\/|$)/.test(page) && !isDynamicRoute(page)) {
page = `/index${page}`
}
// Resolve on anything that doesn't start with `/`
if (!page.startsWith('/')) {
page = `/${page}`
}
// Throw when using ../ etc in the pathname
const resolvedPage = posix.normalize(page)
if (page !== resolvedPage) {
throw new Error(
`Requested and resolved page mismatch: ${page} ${resolvedPage}`
)
}
return page
}
...@@ -55,8 +55,8 @@ import { ...@@ -55,8 +55,8 @@ import {
loadGetInitialProps, loadGetInitialProps,
} from '../shared/lib/utils' } from '../shared/lib/utils'
import { HtmlContext } from '../shared/lib/html-context' import { HtmlContext } from '../shared/lib/html-context'
import { denormalizePagePath } from './denormalize-page-path' import { normalizePagePath } from '../shared/lib/page-path/normalize-page-path'
import { normalizePagePath } from './normalize-page-path' import { denormalizePagePath } from '../shared/lib/page-path/denormalize-page-path'
import { getRequestMeta, NextParsedUrlQuery } from './request-meta' import { getRequestMeta, NextParsedUrlQuery } from './request-meta'
import { import {
allowedStatusCodes, allowedStatusCodes,
......
...@@ -7,11 +7,12 @@ import { ...@@ -7,11 +7,12 @@ import {
SERVER_DIRECTORY, SERVER_DIRECTORY,
SERVERLESS_DIRECTORY, SERVERLESS_DIRECTORY,
} from '../shared/lib/constants' } from '../shared/lib/constants'
import { normalizePagePath, denormalizePagePath } from './normalize-page-path'
import { normalizeLocalePath } from '../shared/lib/i18n/normalize-locale-path' import { normalizeLocalePath } from '../shared/lib/i18n/normalize-locale-path'
import { normalizePagePath } from '../shared/lib/page-path/normalize-page-path'
import { denormalizePagePath } from '../shared/lib/page-path/denormalize-page-path'
import type { PagesManifest } from '../build/webpack/plugins/pages-manifest-plugin' import type { PagesManifest } from '../build/webpack/plugins/pages-manifest-plugin'
import type { MiddlewareManifest } from '../build/webpack/plugins/middleware-plugin' import type { MiddlewareManifest } from '../build/webpack/plugins/middleware-plugin'
import type { WasmBinding } from '../build/webpack/loaders/next-middleware-wasm-loader' import type { WasmBinding } from '../build/webpack/loaders/get-module-build-info'
export function pageNotFoundError(page: string): Error { export function pageNotFoundError(page: string): Error {
const err: any = new Error(`Cannot find module for page: ${page}`) const err: any = new Error(`Cannot find module for page: ${page}`)
......
...@@ -10,7 +10,7 @@ import { ...@@ -10,7 +10,7 @@ import {
AbortSignal, AbortSignal,
} from 'next/dist/compiled/abort-controller' } from 'next/dist/compiled/abort-controller'
import vm from 'vm' import vm from 'vm'
import type { WasmBinding } from '../../../build/webpack/loaders/next-middleware-wasm-loader' import type { WasmBinding } from '../../../build/webpack/loaders/get-module-build-info'
const WEBPACK_HASH_REGEX = const WEBPACK_HASH_REGEX =
/__webpack_require__\.h = function\(\) \{ return "[0-9a-f]+"; \}/g /__webpack_require__\.h = function\(\) \{ return "[0-9a-f]+"; \}/g
......
import type { WasmBinding } from '../../../build/webpack/loaders/next-middleware-wasm-loader' import type { WasmBinding } from '../../../build/webpack/loaders/get-module-build-info'
import type { RequestData, FetchEventResult } from '../types' import type { RequestData, FetchEventResult } from '../types'
import { getModuleContext } from './context' import { getModuleContext } from './context'
......
type Flattened<T> = T extends Array<infer U> ? Flattened<U> : T
/**
* Returns a new list by pulling every item out of it (and all its sub-arrays)
* and putting them in a new array, depth-first. Stolen from Ramda.
*/
export function flatten<T extends readonly any[]>(list: T): Flattened<T>[] {
let jlen: number,
j: number,
value,
idx = 0,
result = []
while (idx < list.length) {
if (Array.isArray(list[idx])) {
value = flatten(list[idx])
j = 0
jlen = value.length
while (j < jlen) {
result[result.length] = value[j]
j += 1
}
} else {
result[result.length] = list[idx]
}
idx += 1
}
return result as Flattened<T>[]
}
import { acceptLanguage } from '../../../server/accept-header' import { acceptLanguage } from '../../../server/accept-header'
import { denormalizePagePath } from '../../../server/denormalize-page-path' import { denormalizePagePath } from '../page-path/denormalize-page-path'
import { detectDomainLocale } from './detect-domain-locale' import { detectDomainLocale } from './detect-domain-locale'
import { formatUrl } from '../router/utils/format-url' import { formatUrl } from '../router/utils/format-url'
import { normalizeLocalePath } from './normalize-locale-path' import { normalizeLocalePath } from './normalize-locale-path'
......
import { ensureLeadingSlash } from './ensure-leading-slash'
import { normalizePathSep } from './normalize-path-sep'
import { relative } from '../isomorphic/path'
import { removePagePathTail } from './remove-page-path-tail'
/**
* Given the absolute path to the pages folder, an absolute file path for a
* page and the page extensions, this function will return the page path
* relative to the pages folder. It doesn't consider index tail. Example:
* - `/Users/rick/my-project/pages/foo/bar/baz.js` -> `/foo/bar/baz`
*
* @param pagesDir Absolute path to the pages folder.
* @param filepath Absolute path to the page.
* @param extensions Extensions allowed for the page.
*/
export function absolutePathToPage(
pagesDir: string,
pagePath: string,
extensions: string[]
) {
return removePagePathTail(
normalizePathSep(ensureLeadingSlash(relative(pagesDir, pagePath))),
extensions
)
}
import { isDynamicRoute } from '../router/utils'
import { normalizePathSep } from './normalize-path-sep'
/**
* Performs the opposite transformation of `normalizePagePath`. Note that
* this function is not idempotent either in cases where there are multiple
* leading `/index` for the page. Examples:
* - `/index` -> `/`
* - `/index/foo` -> `/foo`
* - `/index/index` -> `/index`
*/
export function denormalizePagePath(page: string) {
let _page = normalizePathSep(page)
return _page.startsWith('/index/') && !isDynamicRoute(_page)
? _page.slice(6)
: _page !== '/index'
? _page
: '/'
}
/**
* For a given page path, this function ensures that there is a leading slash.
* If there is not a leading slash, one is added, otherwise it is noop.
*/
export function ensureLeadingSlash(path: string) {
return path.startsWith('/') ? path : `/${path}`
}
import { denormalizePagePath } from './denormalize-page-path'
import { flatten } from '../flatten'
import { join } from '../isomorphic/path'
/**
* Calculate all possible pagePaths for a given normalized pagePath along with
* allowed extensions. This can be used to check which one of the files exists
* and to debug inspected locations.
*
* @param normalizedPagePath Normalized page path (it will denormalize).
* @param extensions Allowed extensions.
*/
export function getPagePaths(normalizedPagePath: string, extensions: string[]) {
const page = denormalizePagePath(normalizedPagePath)
return flatten(
extensions.map((extension) => {
return !normalizedPagePath.endsWith('/index')
? [`${page}.${extension}`, join(page, `index.${extension}`)]
: [join(page, `index.${extension}`)]
})
)
}
import { ensureLeadingSlash } from './ensure-leading-slash'
import { isDynamicRoute } from '../router/utils'
import { posix } from '../isomorphic/path'
/**
* Takes a page and transforms it into its file counterpart ensuring that the
* output is normalized. Note this function is not idempotent because a page
* `/index` can be referencing `/index/index.js` and `/index/index` could be
* referencing `/index/index/index.js`. Examples:
* - `/` -> `/index`
* - `/index/foo` -> `/index/index/foo`
* - `/index` -> `/index/index`
*/
export function normalizePagePath(page: string): string {
const normalized = ensureLeadingSlash(
/^\/index(\/|$)/.test(page) && !isDynamicRoute(page)
? `/index${page}`
: page === '/'
? '/index'
: page
)
const resolvedPage = posix.normalize(normalized)
if (resolvedPage !== normalized) {
throw new Error(
`Requested and resolved page mismatch: ${normalized} ${resolvedPage}`
)
}
return normalized
}
/**
* For a given page path, this function ensures that there is no backslash
* escaping slashes in the path. Example:
* - `foo\/bar\/baz` -> `foo/bar/baz`
*/
export function normalizePathSep(path: string): string {
return path.replace(/\\/g, '/')
}
import { normalizePathSep } from './normalize-path-sep'
/**
* Removes the file extension for a page and the trailing `index` if it exists
* making sure to not return an empty string. The page head is not touched
* and returned as it is passed. Examples:
* - `/foo/bar/baz/index.js` -> `/foo/bar/baz`
* - `/foo/bar/baz.js` -> `/foo/bar/baz`
*
* @param pagePath A page to a page file (absolute or relative)
* @param extensions Extensions allowed for the page.
*/
export function removePagePathTail(pagePath: string, extensions: string[]) {
return (
normalizePathSep(pagePath)
.replace(new RegExp(`\\.+(?:${extensions.join('|')})$`), '')
.replace(/\/index$/, '') || '/'
)
}
...@@ -18,7 +18,7 @@ import { ...@@ -18,7 +18,7 @@ import {
} from '../../../client/route-loader' } from '../../../client/route-loader'
import { handleClientScriptLoad } from '../../../client/script' import { handleClientScriptLoad } from '../../../client/script'
import isError, { getProperError } from '../../../lib/is-error' import isError, { getProperError } from '../../../lib/is-error'
import { denormalizePagePath } from '../../../server/denormalize-page-path' import { denormalizePagePath } from '../page-path/denormalize-page-path'
import { normalizeLocalePath } from '../i18n/normalize-locale-path' import { normalizeLocalePath } from '../i18n/normalize-locale-path'
import mitt from '../mitt' import mitt from '../mitt'
import { import {
......
...@@ -75,12 +75,12 @@ describe.skip('Production Usage with swcMinify', () => { ...@@ -75,12 +75,12 @@ describe.skip('Production Usage with swcMinify', () => {
expect(serverTrace.version).toBe(1) expect(serverTrace.version).toBe(1)
expect( expect(
serverTrace.files.some((file) => serverTrace.files.some((file) =>
file.includes('next/dist/server/send-payload/index.js') file.includes('next/dist/shared/lib/page-path/normalize-page-path.js')
) )
).toBe(true) ).toBe(true)
expect( expect(
serverTrace.files.some((file) => serverTrace.files.some((file) =>
file.includes('next/dist/server/normalize-page-path.js') file.includes('next/dist/shared/lib/page-path/normalize-page-path.js')
) )
).toBe(true) ).toBe(true)
expect( expect(
......
...@@ -105,7 +105,7 @@ describe('Production Usage', () => { ...@@ -105,7 +105,7 @@ describe('Production Usage', () => {
).toBe(true) ).toBe(true)
expect( expect(
serverTrace.files.some((file) => serverTrace.files.some((file) =>
file.includes('next/dist/server/normalize-page-path.js') file.includes('next/dist/shared/lib/page-path/normalize-page-path.js')
) )
).toBe(true) ).toBe(true)
expect( expect(
......
/* eslint-env jest */ /* eslint-env jest */
import { findPageFile } from 'next/dist/server/lib/find-page-file' import { findPageFile } from 'next/dist/server/lib/find-page-file'
import { normalizePagePath } from 'next/dist/server/normalize-page-path' import { normalizePagePath } from 'next/dist/shared/lib/page-path/normalize-page-path'
import { join } from 'path' import { join } from 'path'
......
/* eslint-env jest */ /* eslint-env jest */
import { join } from 'path' import { join } from 'path'
import { normalizePagePath } from 'next/dist/server/normalize-page-path'
import { SERVER_DIRECTORY, CLIENT_STATIC_FILES_PATH } from 'next/constants' import { SERVER_DIRECTORY, CLIENT_STATIC_FILES_PATH } from 'next/constants'
import { normalizePagePath } from 'next/dist/shared/lib/page-path/normalize-page-path'
import { import {
requirePage, requirePage,
getPagePath, getPagePath,
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment