{"id":106,"date":"2026-09-24T15:11:53","date_gmt":"2026-09-24T15:11:53","guid":{"rendered":"https:\/\/evoxia.online\/?p=106"},"modified":"2026-09-24T15:11:53","modified_gmt":"2026-09-24T15:11:53","slug":"graph-neural-networks-deep-learning-for-data-made-of-connections","status":"publish","type":"post","link":"https:\/\/evoxia.online\/?p=106","title":{"rendered":"Graph Neural Networks: Deep Learning for Data Made of Connections"},"content":{"rendered":"\n

Data That Does Not Fit in a Grid<\/strong><\/p>\n\n\n\n

Convolutional networks work on images because an image is a regular grid: every pixel has the same number of neighbors, arranged in the same way. Recurrent networks and transformers work on text because a sentence is a sequence with a clear order. A lot of important data has neither structure.<\/p>\n\n\n\n

A molecule is a set of atoms connected by bonds. A social network is a set of people connected by friendships. A road network is a set of intersections connected by streets. A citation network, a power grid, a supply chain, a protein’s contact map: all of them are graphs, made of nodes and the edges between them. The number of neighbors varies from node to node, there is no natural ordering of the nodes, and the connections themselves carry most of the information. Graph neural networks (GNNs) are the family of models built for this kind of data.<\/p>\n\n\n\n

Message Passing<\/strong><\/p>\n\n\n\n

Most GNNs follow one pattern, which Justin Gilmer and colleagues formalized in 2017 as message passing. Each node starts with a feature vector, for example the element type and charge of an atom. In each layer, every node collects messages from its neighbors, combines them with a function that does not depend on their order (such as a sum, mean or max), and uses the result to update its own vector.<\/p>\n\n\n\n

After one layer, each node’s representation contains information about its direct neighbors. After two layers, it includes neighbors of neighbors. After k layers, each node has a summary of its k-hop neighborhood. The same learned functions are applied at every node, similar to how a convolutional filter is applied at every position in an image, so the model works on graphs of any size and shape.<\/p>\n\n\n\n

The node representations can then be used in three ways. Node-level tasks predict something about each node, such as the topic of a paper in a citation network. Edge-level tasks predict whether a connection exists or what kind it is, such as whether a user will interact with an item. Graph-level tasks pool all node vectors into one and predict a property of the whole graph, such as whether a molecule is toxic.<\/p>\n\n\n\n

From the First GNNs to GCN<\/strong><\/p>\n\n\n\n

Franco Scarselli and colleagues described a graph neural network model in 2009, in which node states were updated repeatedly until they reached a fixed point. A different line of work started from spectral graph theory. Bruna and colleagues in 2014 defined convolution on graphs through the eigenvectors of the graph Laplacian, the graph equivalent of Fourier analysis. It was mathematically elegant but expensive and tied to one specific graph.<\/p>\n\n\n\n

Thomas Kipf and Max Welling simplified this in their 2017 ICLR paper on graph convolutional networks (GCN). They approximated the spectral convolution so heavily that each layer became a simple operation: average the features of each node and its neighbors, with a normalization based on node degrees, then apply a linear transformation and a nonlinearity. It was cheap, easy to implement, and gave strong results on semi-supervised node classification in citation networks, where only a small fraction of nodes have labels. GCN is still the model most people learn first.<\/p>\n\n\n\n

GraphSAGE, Attention and Expressive Power<\/strong><\/p>\n\n\n\n

GCN as originally described needs the whole graph at training time. GraphSAGE, from William Hamilton, Rex Ying and Jure Leskovec (2017), made GNNs work on very large and changing graphs by sampling a fixed number of neighbors for each node and learning aggregation functions that apply to nodes never seen during training.<\/p>\n\n\n\n

Graph attention networks (GAT), from Petar Veli\u010dkovi\u0107 and colleagues in 2018, replaced the fixed averaging weights with learned attention: each node decides how much to listen to each neighbor, based on both of their features. This helps when some neighbors are much more relevant than others.<\/p>\n\n\n\n

A theoretical question followed: which graphs can a GNN tell apart? Keyulu Xu and colleagues showed in 2019 that standard message-passing GNNs are at most as powerful as the Weisfeiler-Lehman graph isomorphism test, a classic algorithm that iteratively relabels nodes based on their neighbors’ labels. There are simple pairs of non-identical graphs that the test, and therefore these GNNs, cannot distinguish. They proposed the Graph Isomorphism Network (GIN), which uses sum aggregation with a multilayer perceptron and reaches that upper bound. Much later research has tried to go beyond it by adding substructure counts, positional encodings, or higher-order message passing.<\/p>\n\n\n\n

Known Weaknesses<\/strong><\/p>\n\n\n\n

Stacking many GNN layers does not work as well as stacking many CNN layers. The first reason is over-smoothing: each layer averages node features with their neighbors, and after many layers all nodes in a connected region end up with nearly the same representation, losing the information that distinguished them. The second is over-squashing: information from an exponentially growing neighborhood must be compressed into a fixed-size vector, so signals from distant nodes get lost. For these reasons most practical GNNs are shallow, often two to four layers.<\/p>\n\n\n\n

Many early models also assumed homophily, the tendency of connected nodes to be similar. That holds in citation networks, where papers cite related papers, but not in graphs where edges connect different kinds of things, such as fraudsters and their victims. Scaling is another challenge, since a node’s neighborhood can explode in size after a few hops in graphs with billions of edges. Graph transformers, which let every node attend to every other node and add information about the graph structure through positional encodings, are one response to over-squashing, at a higher computational cost.<\/p>\n\n\n\n

Molecules and Drug Discovery<\/strong><\/p>\n\n\n\n

Chemistry is one of the most natural applications, because a molecule is a graph. Gilmer and colleagues introduced their message passing framework while predicting quantum chemical properties of small molecules, reaching accuracy close to that of expensive quantum calculations at a fraction of the cost.<\/p>\n\n\n\n

In 2020, Jonathan Stokes and colleagues at MIT published in Cell the discovery of halicin, an antibiotic identified with a message-passing neural network trained to predict whether molecules inhibit the growth of E. coli. After screening a drug repurposing library and then more than 100 million molecules from a larger database, they found a compound structurally different from existing antibiotics that was effective against several drug-resistant bacteria in lab and mouse experiments. GNNs are also used to predict material properties, and to reason about protein structure, where residues in contact form a natural graph.<\/p>\n\n\n\n

Maps, Weather and Recommendations<\/strong><\/p>\n\n\n\n

In 2020, DeepMind described a GNN deployed in Google Maps to predict travel times. The road network was divided into \u00absupersegments\u00bb of adjacent roads that share traffic, and the model learned how congestion spreads between them. According to DeepMind, it improved the accuracy of real-time arrival estimates by up to 50% in cities such as Berlin, Jakarta, S\u00e3o Paulo, Sydney, Tokyo and Washington D.C.<\/p>\n\n\n\n

GraphCast, published by Remi Lam and colleagues in Science in 2023, uses a GNN to forecast global weather up to ten days ahead. The Earth’s surface is represented as a multi-resolution mesh of nodes, and message passing over that mesh moves information around the globe. It outperformed the European Centre for Medium-Range Weather Forecasts’ high-resolution deterministic system (HRES) on 90% of 1,380 verification targets, and it produces a forecast in under a minute on a single TPU, compared to hours on a supercomputer for traditional numerical models.<\/p>\n\n\n\n

Recommendation systems are another major use, since users and items form a bipartite graph. PinSage, from Rex Ying and colleagues at Pinterest in 2018, applied GraphSAGE-style convolutions to a graph with billions of nodes to recommend pins. Banks and payment companies use GNNs for fraud detection, since fraudulent accounts often reveal themselves through their connections more than through their individual features.<\/p>\n\n\n\n

Reflection<\/strong><\/p>\n\n\n\n

The main idea behind graph neural networks is that a node is described by its neighborhood as much as by its own features. An atom’s role depends on what it is bonded to, and a road’s traffic depends on the roads feeding into it. Convolutional networks relied on the same idea, with the neighborhood fixed by the pixel grid. GNNs let the data define the neighborhood, which is why the same kind of model can work on molecules, road networks and the whole planet’s atmosphere.<\/p>\n\n\n\n

By: Max Johnson B.<\/p>\n","protected":false},"excerpt":{"rendered":"

Data That Does Not Fit in a Grid Convolutional networks work on images because an image is a regular grid: every pixel has the same number of neighbors, arranged in the same way. Recurrent networks and transformers work on text because a sentence is a sequence with a clear order. A lot of important data […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-106","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/evoxia.online\/index.php?rest_route=\/wp\/v2\/posts\/106","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/evoxia.online\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/evoxia.online\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/evoxia.online\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/evoxia.online\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=106"}],"version-history":[{"count":0,"href":"https:\/\/evoxia.online\/index.php?rest_route=\/wp\/v2\/posts\/106\/revisions"}],"wp:attachment":[{"href":"https:\/\/evoxia.online\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=106"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/evoxia.online\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=106"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/evoxia.online\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=106"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}