File System (fs)

Why Node Has a File System Module

A backend server constantly touches files: reading a config, writing logs, saving uploads, generating reports. The built-in fs module is Node's bridge to the file system.

It's a core module — no install needed:

import fs from 'node:fs/promises'; // modern, Promise-based
// or
const fs = require('fs');           // classic, callback/sync

Always prefer node:fs/promises for new code. The other styles still work, but async/await is the cleanest.

Three API Flavours (Pick One)

const fs = require('fs');
const fsp = require('fs').promises;
// 1. Synchronous — blocks the event loop. Avoid in servers.
const data = fs.readFileSync('config.json', 'utf8');
// 2. Callback — old style, still common in tutorials
fs.readFile('config.json', 'utf8', (err, data) => { /* … */ });
// 3. Promise / async-await — recommended
const data2 = await fsp.readFile('config.json', 'utf8');

Rule of thumb: *Sync versions are fine in CLI scripts and startup code. They are *poison* inside a request handler — they freeze every other user.

Daily-Use Recipes

import fs from 'node:fs/promises';
import path from 'node:path';
// Read text
const text = await fs.readFile('notes.txt', 'utf8');
// Write text (overwrites)
await fs.writeFile('notes.txt', 'hello');
// Append (don't overwrite)
await fs.appendFile('app.log', 'request received\n');
// Check existence
try { await fs.access('config.json'); } catch { /* missing */ }
// Make folder (parents too)
await fs.mkdir('uploads/avatars', { recursive: true });
// List a folder
const items = await fs.readdir('./src');
// Delete
await fs.unlink('temp.txt');           // file
await fs.rm('build', { recursive: true, force: true }); // folder

Always Use path for Filenames

import path from 'node:path';
const file = path.join(__dirname, 'data', 'users.json');

Why: Windows uses \, Linux/macOS use /. path.join picks the right one. Hard-coding '/' quietly breaks on Windows.

Beginner Mistakes to Skip

1. Forgetting 'utf8'. Without an encoding, readFile returns a raw Buffer (bytes), not a string. JSON.parse on a Buffer throws a confusing error. 2. Using *Sync inside a request handler. Looks fine in dev with one user, falls over in prod. 3. Joining paths with string concat ('./folder' + name). Use path.join. 4. No try/catch around async fs calls. A missing file becomes an unhandled promise rejection. 5. Storing user uploads in your repo's source folder. Use a dedicated uploads/ folder (gitignored) or, better, object storage like S3.

Intermediate: Streams — The Big-File Solution

Reading a 2 GB log file with readFile would load *all* 2 GB into memory and crash your server. Streams read/write in tiny chunks (default 64 KB):

import { createReadStream } from 'node:fs';
const stream = createReadStream('huge.log', { encoding: 'utf8' });
stream.on('data', (chunk) => console.log(got ${chunk.length} chars));
stream.on('end',  () => console.log('done'));
stream.on('error',(e) => console.error(e));

Memory usage stays flat no matter how big the file is.

Intermediate: pipe — Connect Streams Together

import { createReadStream, createWriteStream } from 'node:fs';
import { createGzip } from 'node:zlib';
// Copy a 10 GB file with constant memory:
createReadStream('movie.mp4').pipe(createWriteStream('backup.mp4'));
// Gzip on the fly:
createReadStream('app.log')
  .pipe(createGzip())
  .pipe(createWriteStream('app.log.gz'));

This pattern powers most file-upload, file-download, and log-rotation libraries you'll ever use.

Intermediate: Modern Stream Pipeline

import { pipeline } from 'node:stream/promises';
import { createReadStream, createWriteStream } from 'node:fs';
import { createGzip } from 'node:zlib';
await pipeline(
  createReadStream('app.log'),
  createGzip(),
  createWriteStream('app.log.gz')
);

pipeline properly handles errors and cleanup across every stream in the chain — much safer than chaining raw .pipe() calls.

Intermediate: JSON Files

// Read
const raw = await fs.readFile('config.json', 'utf8');
const config = JSON.parse(raw);
// Modify + write
config.lastRun = new Date().toISOString();
await fs.writeFile('config.json', JSON.stringify(config, null, 2));

For a real database use Postgres/MongoDB — JSON files are great for config, dev fixtures, and tiny tools.

Intermediate: Watching Files for Changes

import { watch } from 'node:fs';
watch('./src', { recursive: true }, (eventType, filename) => {
  console.log(eventType, filename); // 'change' or 'rename'
});

Useful for build tools and hot reload. For production-grade watching across platforms, libraries like chokidar paper over OS quirks.

Advanced: Buffers — Raw Bytes

Not everything is text. Images, audio, encrypted blobs are binary. Node represents them as Buffers:

const img = await fs.readFile('logo.png');  // no encoding → Buffer
console.log(img.length, 'bytes');
const base64 = img.toString('base64');

Buffers behave like Uint8Array and can be sliced, concatenated, and converted between encodings ('utf8', 'hex', 'base64').

Advanced: File Handles for Random Access

const fh = await fs.open('big.bin', 'r');
const { buffer, bytesRead } = await fh.read(Buffer.alloc(1024), 0, 1024, 5000);
await fh.close();

Lets you read/write at arbitrary offsets — needed for binary file formats, custom databases, or memory-mapped patterns.

Advanced: Cross-Platform Pitfalls

• Path separators: path.sep ('/' vs '\\').
• Case sensitivity: USERS.txt and users.txt are the same file on Windows/macOS, different on Linux.
• Line endings: '\n' vs '\r\n'. Use os.EOL if generating files for the host system.
• Permissions: fs.chmod, fs.chown are largely no-ops on Windows.

Advanced: When *Not* to Touch the Disk

For uploads in any serious app, write straight to object storage (S3, R2, GCS) and only keep a URL in your DB. Local disk is:

• Lost on container restart.
• Not shared across instances.
• Slow vs network-attached storage at scale.

Use fs for: config, logs, temp files, build outputs, dev fixtures.

Practice Path

1. Write a script that reads names.txt, lowercases every line, and writes to names-lower.txt — using fs/promises and utf8. 2. Create a CLI that gzips any file path passed in process.argv[2] using pipeline + streams. 3. Build a tiny log-rotator: append to app.log; once it crosses 1 MB, rename to app-${Date.now()}.log and start fresh. 4. Watch a folder and print the size (in KB) every time a file inside changes.