## Stumbling across the error
Last night I was working on adding the ability to dynamically generate og:images for this site. Following some [similar](https://github.com/samrobbins85/next-og-image) [implementations](https://github.com/juice49/next-og-image), I decided to use [puppeteer-core](https://www.npmjs.com/package/puppeteer-core) and [chrome-aws-lambda](https://www.npmjs.com/package/chrome-aws-lambda) to take a snapshot of a [page that I built](https://github.com/zephraph/just-be/blob/master/pages/og/posts/%5Bslug%5D.tsx) that acts as a preview of the og:image. Once I got it set up, I ran across a weird error trying to deploy Vercel.
>[!error]
> The Serverless Function "api/feed/atom" is 50.38mb which exceeds the maximum size limit of 50mb. Learn More: https://vercel.link/serverless-function-size
This error was weird to me because I know my atom feed API route (and its dependencies) are nowhere near `50 mb`. The learn more link to the error description essentially says what's already apparent: You can't have lambdas that big. This lambda _isn't_ that big though, right?
The only possible culprit for this would have to be puppeteer. None of my other dependencies are remotely that big. The _weird_ part is that the feed API doesn't import puppeteer. On top of that Vercel's output for the atom feed API in the build summary said it was only supposed to be `~70 kb`. While trying to reason through all this, I noticed the following line above the error message in the logs.
```
Traced Next.js server files in: 15500.049ms
```
I took two things from this
1. Vercel is analyzing the all the lambdas' dependencies to see what should be included for server execution.
2. Given that it took so long and the bundle was reported to be so large, they may actually be bundling _all_ the server dependencies together and shipping that for each lambda. Maybe via a [lambda layer](https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html)?
While the exact mechanics weren't clear to me, I had enough to go on to start digging through GitHub for answers. My first question was simple: how do they trace the server dependencies?
## Tracing the clues
After searching around in their GitHub, I stumbled across [vercel/nft](https://github.com/vercel/nft) (Node File Trace). What's the first thing in the readme?
> This package is used in @vercel/node and @vercel/next to determine exactly which files (including node_modules) are necessary for the application runtime.
bingo.
The API for this library is incredibly simple. Give it a list of files to analyze and it'll return a list of all those files dependencies.
```jsx
const { nodeFileTrace } = require('@vercel/nft');
const files = ['./src/main.js', './src/second.js'];
const { fileList } = await nodeFileTrace(files);
```
All I needed then was the list of files to trace. The tracing step in the logs is listed right after Next's build step, so the build output directory seemed like a good place to look.
Next generates a `.next` directory anytime you run `next build`. My hope was to find something to key me off in there.
![[next-build-output.png]]
_Next's build output_
Turns out that hope was well founded. Inside of .next there's a server directory containing all the resources for running things server side. There's a pages-manifest.json file that looked like exactly what I was looking for. Also note that there's a pages directory which contains all the server modules compiled and ready to go.
```json
{
"/_app": "pages/_app.js",
"/_document": "pages/_document.js",
"/_layout": "pages/_layout.html",
"/about": "pages/about.js",
"/api/feed/atom": "pages/api/feed/atom.js",
"/api/feed/json": "pages/api/feed/json.js",
"/api/feed/rss": "pages/api/feed/rss.js",
"/api/post/[slug]": "pages/api/post/[slug].js",
"/": "pages/index.js",
"/og/posts/[slug]": "pages/og/posts/[slug].js",
"/posts/[post]": "pages/posts/[post].js",
"/posts/_layout": "pages/posts/_layout.html",
"/tips": "pages/tips.js",
"/_error": "pages/_error.js",
"/404": "pages/404.html"
}
```
The first thing I wanted to know here was if somehow puppeteer was accidentally getting bundled up into the atom feed lambda. I wrote a out a little script just to test that.
```js
const { nodeFileTrace } = require('@vercel/nft')
nodeFileTrace(['.next/server/pages/api/feed/atom.js'])
.then(({ fileList }) => fileList
.forEach((file) => console.log(file)) )
```
Running this confirmed what I had already expected: No puppeteer dependency. At this point, I'm relatively confident that my earlier assumption that everything is bundled together is true. I wanted to go a little further though and have a detailed log of all the dependencies, their sizes, and the total size of the output. To achieve that goal I wrote this [trace script](https://github.com/zephraph/just-be/blob/master/scripts/trace.js). It passes the values of the server manifest file to node file trace, maps through the results and uses fs.stat to get the file size for each, does some formatting to make it readable, and dumps it to a file called trace-debug.log. The results were... revealing.
## The fruits of our labor
I won't dump the whole 1200 lines of files being bundled together here, but the top of the log says enough.
```
Total size: 55.8 MB
44.0 MB node_modules/chrome-aws-lambda/bin/chromium.br
3.6 MB .next/server/pages/tips.js
1.5 MB node_modules/chrome-aws-lambda/bin/aws.tar.br
991.4 KB node_modules/chrome-aws-lambda/bin/swiftshader.tar.br
529.8 KB node_modules/lodash/lodash.js
267.7 KB node_modules/framer-motion/dist/framer-motion.cjs.js
132.7 KB .next/server/pages/posts/[post].js
129.3 KB .next/server/pages/index.js
124.0 KB .next/server/pages/about.js
```
The 55.8 MB is a bit bigger than the 50.38 MB that Vercel showed in the error, but not by much. There's likely something in this list that's getting stripped out. Still... that's a lot. Unsurprisingly chromium is the behemoth. That means that if chrome-aws-lambda (even without puppeteer) is included in your build you've already lost 88% of the code capacity of your server bundle. The entirety of the remainder of your site and all its dependencies needs to fit in 6 MB. If you do something crazy like me and ship prettier in one of your lambdas to format code snippets then you're just SOL.
## Now what?
I plan to release a more generic version of [the script](https://github.com/zephraph/just-be/blob/master/scripts/trace.js) at some point, but it should work well enough if you just plop it in your project now. This at least helps reveal what's been hard to debug up to this point.
The obvious next question is probably this... how do we fix it? I just thought I'd need to host the puppeteer rendering somewhere else. [Björn Rave](https://twitter.com/bjoern_rave) showed me otherwise.
<blockquote class="twitter-tweet"><p lang="en" dir="ltr">the "trick" is to put the lambda with puppeteer in a api/ folder in the root. that way it will be built on it's own and can use 50mb just for itself</p>— Björn Rave (@bjoern_rave) <a href="https://twitter.com/bjoern_rave/status/1375860916386983936?ref_src=twsrc%5Etfw">March 27, 2021</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
So, your tip of the day. While Next.js bundles all it's server dependencies together, any API endpoints living at the root in api/ will be bundled separately. These functions are [Vercel's serverless functions](https://vercel.com/docs/serverless-functions/introduction) which don't have all the niceties of Next's functions, but are individually bundled. I've got that working on my site now if you want to check it out. If folks are interested I could write a guide for that.
## Takeaways for the Vercel crew
If there's any of the Vercel crew reading this, I do have a few takeaways that I think would make this experience better.
1. While I understand that how lambda dependencies are bundled is an implementation detail that most folks don't necessarily need to know about, it's also a leaky abstraction. It may be worth putting some details in the docs about how Next bundles its server dependencies on Vercel and linking to that instead of just the generic lambda size error docs if the lambda size for Next is too big.
2. There should absolutely be warnings in the build logs when a build is getting close to running over the 50mb limit. I think it'd also be useful just to list out the top 5 or 10 files by size to inform the user what's contributing to the bloat. Likewise if it errors out... list the top files and their sizes as possible causes.