Skip to main content

What are streams in Node.js and how do they work?

Streams in Node.js are a powerful way to handle reading and writing data efficiently, especially when dealing with large amounts of data. Instead of loading an entire file or buffer into memory at once, streams allow you to process data in chunks as it's being read or written, making them ideal for large files or data sources.

                                           

Types of Streams in Node.js

  1. Readable Streams: These allow you to read data, one chunk at a time.

    • Examples: fs.createReadStream(), http.IncomingMessage, process.stdin

  2. Writable Streams: These allow you to write data, one chunk at a time.

    • Examples: fs.createWriteStream(), http.ServerResponse, process.stdout

  3. Duplex Streams: These are both readable and writable.

    • Examples: net.Socket, tls.TLSSocket

  4. Transform Streams: These are a type of duplex stream that modifies data as it's being read and written.

    • Examples: zlib.createGzip() (for compression), crypto.createCipher() (for encryption)

How Streams Work

1. Readable Stream

A readable stream reads data from a source (e.g., a file or HTTP request) and makes it available in small chunks.

const fs = require('fs');
const readStream = fs.createReadStream('largeFile.txt', 'utf8');

readStream.on('data', chunk => {
  console.log('Received chunk:', chunk); // Process each chunk
});

readStream.on('end', () => {
  console.log('File reading completed!');
});

  • 'data' event: Fired when a chunk of data is available to read.

  • 'end' event: Fired when the stream has no more data to read.

2. Writable Stream

A writable stream writes data to a destination (e.g., a file or HTTP response) in small chunks.

const fs = require('fs');
const writeStream = fs.createWriteStream('output.txt');

writeStream.write('Hello, ');
writeStream.write('world!');
writeStream.end(() => {
  console.log('Writing completed!');
});

  • write() method: Used to send data to the writable stream.

  • end() method: Signals that no more data will be written.

3. Piping Streams

You can connect a readable stream to a writable stream using the pipe() method. This allows data to flow from one stream to another automatically, which is very efficient for tasks like reading from a file and writing to another file.

const fs = require('fs');

const readStream = fs.createReadStream('input.txt');
const writeStream = fs.createWriteStream('output.txt');

// Pipe the data from the read stream to the write stream
readStream.pipe(writeStream);

  • pipe() method: Automatically manages the flow of data from the source (readable) to the destination (writable).

Advantages of Streams

  • Efficiency: Instead of loading everything into memory, streams process data in chunks, which is ideal for large files or real-time data.

  • Low Memory Usage: Since streams don’t store the entire data set in memory, they’re much more memory efficient, which is particularly useful for server-side applications.

  • Real-Time Data Processing: Streams are useful for real-time data flows like audio, video, or any form of continuous input/output (e.g., HTTP requests/responses).

Real-World Use Cases for Streams

  • File Manipulation: Reading and writing large files without consuming too much memory.

  • Pipes in Networking: Streaming data between servers (e.g., downloading/uploading files).

  • Real-time Communication: Streaming audio/video data, real-time logs, etc.

  • Data Transformation: Compressing or encrypting data on the fly (using transform streams).

Popular posts from this blog

Explain the Angular compilation process: View Engine vs. Ivy.

 The Angular compilation process transforms your Angular templates and components into efficient JavaScript code that the browser can execute. Over time, Angular has evolved from the View Engine compiler to a newer, more efficient system called Ivy . Here's a breakdown of the differences between View Engine and Ivy , and how each affects the compilation process: 🔧 1. What Is Angular Compilation? Angular templates ( HTML inside components) are not regular HTML—they include Angular-specific syntax like *ngIf , {{ }} interpolation, and custom directives. The compiler translates these templates into JavaScript instructions that render and update the DOM. Angular uses Ahead-of-Time (AOT) or Just-in-Time (JIT) compilation modes: JIT : Compiles in the browser at runtime (used in development). AOT : Compiles at build time into efficient JS (used in production). 🧱 2. View Engine (Legacy Compiler) ➤ Used in Angular versions < 9 🔍 How It Works: Compiles templat...
Read more

Explain the concept of ControlValueAccessor in custom form components.

 In Angular, the ControlValueAccessor interface is what allows custom form components to work seamlessly with Angular forms (both reactive and template-driven). 🧠 What is ControlValueAccessor ? It’s an Angular bridge between your custom component and the Angular Forms API . When you use a custom form component (like a date picker, dropdown, slider, etc.), Angular doesn't automatically know how to read or write its value. That’s where ControlValueAccessor comes in. It tells Angular: How to write a value to the component How to notify Angular when the component’s value changes How to handle disabled state 📦 Common Built-in Examples: <input> and <select> already implement ControlValueAccessor You implement it when creating custom form controls 🔧 Key Methods in the Interface Method Purpose writeValue(obj: any) Called by Angular to set the value in the component registerOnChange(fn: any) Passes a function to call when the component value ch...
Read more

What are the different types of directives in Angular? Give real-world examples.

In Angular, directives are classes that allow you to manipulate the DOM or component behavior . There are three main types of directives: 🧱 1. Component Directives Technically, components are directives with a template. They control a section of the screen (UI) and encapsulate logi c. ✅ Example: @Component ({ selector : 'app-user-card' , template : `<h2>{{ name }}</h2>` }) export class UserCardComponent { name = 'Alice' ; } 📌 Real-World Use: A ProductCardComponent showing product details on an e-commerce site. A ChatMessageComponent displaying individual messages in a chat app. ⚙️ 2. Structural Directives These change the DOM layout by adding or removing elements. ✅ Built-in Examples: *ngIf : Conditionally includes a template. *ngFor : Iterates over a list and renders template for each item. *ngSwitch : Switches views based on a condition. 📌 Real-World Use: < div * ngIf = "user.isLoggedIn...
Read more