Skip to main content
Cornell University
We gratefully acknowledge support from
the Simons Foundation and member institutions.
arxiv logo > physics > arXiv:2112.00183

Help | Advanced Search

Physics > Physics and Society

(physics)
[Submitted on 30 Nov 2021 (v1), last revised 10 Jan 2022 (this version, v4)]

Title:Descriptive vs. inferential community detection: pitfalls, myths and half-truths

Authors:Tiago P. Peixoto
Download PDF
Abstract: Community detection is one of the most important methodological fields of network science, and one which has attracted a significant amount of attention over the past decades. This area deals with the automated division of a network into fundamental building blocks, with the objective of providing a summary of its large-scale structure. Despite its importance and widespread adoption, there is a noticeable gap between what is considered the state-of-the-art and the methods that are actually used in practice in a variety of fields. Here we attempt to address this discrepancy by dividing existing methods according to whether they have a "descriptive" or an "inferential" goal. While descriptive methods find patterns in networks based on intuitive notions of community structure, inferential methods articulate a precise generative model, and attempt to fit it to data. In this way, they are able to provide insights into the mechanisms of network formation, and separate structure from randomness in a manner supported by statistical evidence. We review how employing descriptive methods with inferential aims is riddled with pitfalls and misleading answers, and thus should be in general avoided. We argue that inferential methods are more typically aligned with clearer scientific questions, yield more robust results, and should be in many cases preferred. We attempt to dispel some myths and half-truths often believed when community detection is employed in practice, in an effort to improve both the use of such methods as well as the interpretation of their results.
Comments: 56 pages, 17 figures
Subjects: Physics and Society (physics.soc-ph); Social and Information Networks (cs.SI); Data Analysis, Statistics and Probability (physics.data-an); Methodology (stat.ME); Machine Learning (stat.ML)
Cite as: arXiv:2112.00183 [physics.soc-ph]
  (or arXiv:2112.00183v4 [physics.soc-ph] for this version)
  https://doi.org/10.48550/arXiv.2112.00183
arXiv-issued DOI via DataCite

Submission history

From: Tiago Peixoto [view email]
[v1] Tue, 30 Nov 2021 23:57:51 UTC (1,967 KB)
[v2] Wed, 8 Dec 2021 12:16:00 UTC (1,969 KB)
[v3] Tue, 4 Jan 2022 11:08:35 UTC (3,938 KB)
[v4] Mon, 10 Jan 2022 09:57:16 UTC (5,796 KB)
Full-text links:

Download:

  • PDF
  • Other formats
Current browse context:
physics.soc-ph
< prev   |   next >
new | recent | 2112
Change to browse by:
cs
cs.SI
physics
physics.data-an
stat
stat.ME
stat.ML

References & Citations

  • INSPIRE HEP
  • NASA ADS
  • Google Scholar
  • Semantic Scholar
a export bibtex citation Loading...

Bookmark

BibSonomy logo Mendeley logo Reddit logo ScienceWISE logo

Bibliographic and Citation Tools

Bibliographic Explorer (What is the Explorer?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)
Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)
  • About
  • Help
  • contact arXivClick here to contact arXiv Contact
  • subscribe to arXiv mailingsClick here to subscribe Subscribe
  • Copyright
  • Privacy Policy
  • Web Accessibility Assistance
  • arXiv Operational Status
    Get status notifications via email or slack