What are Data Driven Documents?

Data Driven Documents, also referred to as D3, is a JavaScript library for visualizing data using web standards. D3 allows you to bind data to HTML and SVG elements to create interactive data visualizations. For example, you can bind a dataset to circles in an SVG and map data values to circle radius and x/y positions to create a data-driven scatterplot. D3 provides a selection mechanism to select elements in the DOM based on data. This allows you to update elements dynamically in response to user input or new data. It utilizes a "data join" to match data with selected elements.

D3 has scale and axis functions that map data values to visual values like positions, sizes, and colors automatically. This avoids manual scaling calculations. For example, setting up a linear scale to map values to x-coordinates and y-coordinates (X, Y). D3 comes with layout functions like force, tree, & cluster - that generate predefined arrangements based on data. This enables topological visualization of relational data. D3 allows powerful animated transitions when updating visualizations. Things like bars of a bar chart smoothly transition their heights based on data updates.

The key strength of D3 is its flexibility. Data Driven Documents allow both simple and complex data visualization types to be created. From simple bar charts to elaborate network diagrams and geographic maps - all driven off your dataset dynamically!

SVG & Group (g) Elements

SVG

SVG Stands for Scalable Vector Graphics. SVG is an XML-based markup language for describing vector graphics. It allows you to create resolution-independent graphics that can scale smoothly to any size.

• D3 uses SVG as one of the primary graphic elements for visualization. Shapes like circles, rectangles, and paths are drawn using SVG on an HTML webpage.

• SVG elements can be manipulated with JavaScript using D3 to change attributes like position, size, color, etc. in response to data.

• Common SVG elements include circle, rect, path, text, and line. These are the building blocks for most visualizations.

const svg = d3
  .select("body")
  .append("svg")
  .attr("class", "svg-canvas")
  .style("height", "450px")
  .style("width", "800px");

Group (g)

Group elements are referenced with the g syntax. The g element is a container used to group other SVG elements together.

The domain in a D3 scale refers to the input data values that will be mapped to the output range.

• Using g allows you to apply transforms/styles to multiple shapes easily. When we use transforms like translate, and scale, these can be applied to the whole group.

• g helps isolate parts of a graphic. For example, grouping all x-axis labels under 1 g for easy selection/update.

• D3 allows for nesting groups within other groups in the parent-child hierarchy for complex graphics.

 const mainGroup = svg
    .append("g")
    .attr("transform", `translate(${margin.left}, ${margin.bottom})`);

In summary, SVG provides the graphical shapes while g allows logical grouping of those visual components for easier manipulation using D3's data joins and transformations. Together they form core building blocks for creating sophisticated data-driven visualizations dynamically.

Scale, Domain, & Range

Let's take a look at a simple code snippet from a project using D3

  const xScale = d3
    .scaleTime()
    .domain(d3.extent(arr, xData))
    .range([0, graphWidth]);

  const yScale = d3
    .scaleLinear()
    .domain([0, d3.max(arr, yData)])
    .range([graphHeight, 0]);

Even though this is only a few lines of code, a lot is going on here! Let's attempt to break it down piece by piece for a better understanding.

Scale

D3 supports several types of scales for mapping domains to ranges. Our example uses two:

• d3.scaleTime() - Time scale. Useful for temporal data like dates. Supports times, calendars, and formatting.

• d3.scaleLinear() - Linear quantitative scale. Good for continuous numerical data.

• Other D3 scale examples include: d3.scaleLog(), d3.scaleQuantile(), d3.scaleQuantize(), d3.scaleThreshold(), d3.scaleOrdinal()

The key difference between these scales is the input domain mapping. Linear scales map numbers linearly. Time scales map times. Log scales map logarithmically. Ordinal scales map discrete categories. The scale is based on the type of data and desired mapping from the input domain to the output visual range. This helps encode your data appropriately for visualization.

Domain

The domain in a D3 scale refers to the input data values that will be mapped to the output range.

.domain([0, 100])

The domain is the set of possible input values from the raw data that will need to be visually encoded. The scale uses this domain to determine how to map these input values to output visual values (like pixel positions, colors, etc.). By default, the scale uses linear mapping from a domain to an output range. Scales automatically calculate nice tick values within the domain and avoid values too close to min/max so labels are not cut off.

Range

The range in a D3 scale refers to the output visual values that correspond to the input domain values.

.range([0, 1000])

While the domain maps to the raw data values, the range maps to application values in the visualization. Domain is data-specific, while the range is representation-specific, tailored to how values visualize appropriately. It represents the set of possible output values as visual attributes like pixel positions, colors, sizes etc. This visual encoding space is what users actually see rendered on the page.

Utility Methods

Both domain and range accept arrays as perameters, but these can modified using some helpful D3 methods.

• d3.min( ) - Returns minimum value in a data array.

const minValue = d3.min([3, 5, 1, 2]); // minValue = 1

• d3.max( ) - Returns maximum value in a data array.

const maxValue = d3.max([3, 5, 1, 2]); // maxValue = 5

• d3.extent( ) - Returns both min and max as an array.

const extent = d3.extent([3, 5, 1, 2]); // extent = [1, 5]

The benefit of using these utility methods is they work for both numbers and dates and handle sorting, undefined values, etc.

Just a Sample, More to Come...

This is just a small sample of the complexities of D3 and data driven documents! There is far more we can dive in to, keep an eye out for a part two in this series discussing axis, legends, shapes, and so much more!!

But, if you need some data visualization without the headache, here is an awesome D3 library that is fantastic!!

NIVO Simplifies the Process

NIVO is a data visualization library built on top of D3 that provides higher-level chart components. Here's some of the features of NIVO and how it relates to D3:

• NIVO provides react components for commonly used visualizations like bar charts, line charts, heatmaps, networks etc.

• These components handle a lot of complex D3 chart logic under the hood. They have sensible defaults and allow extensive customization through props.

• With NIVO you don't have to code up D3 selections, scales, axes from scratch. Just pass data/config and render a chart.

• NIVO charts handle interactivity like hovering, zooming, etc. right out of the box. Allowing you to focus on data transforms and customizations.

• Because NIVO utilizes D3 for rendering under the hood, you can mix in any custom D3 logic at lower levels if needed.

• NIVO makes it easier and faster to build common data visualization types and greatly reduces boilerplate code compared to using D3 from scratch.

In summary, NIVO provides reusable charts on top of D3's low-level capabilities. It abstracts away some complexity making development easier. But D3's flexibility is still available if needed for highly customized visuals. Together they provide both simplicity and customizability for data visualization.

Doc Links

**As always, do your homework, read the docs, educate yourself!!

• D3

• Observable

• NIVO

• Top 8 JS libraries for Data Vis (blog)