The skills and tools for collecting, verifying and correlating information from different types of systems is an essential skill when tracking down hackers. This book explores Open Source Intelligence Gathering (OSINT) inside out from multiple perspectives, including those of hackers and seasoned intelligence experts. OSINT refers to the techniques and tools required to harvest publicly available data concerning a person or an organization. With several years of experience of tracking hackers with OSINT, the author whips up a classical plot-line involving a hunt for a threat actor. While taking the audience through the thrilling investigative drama, the author immerses the audience with in-depth knowledge of state-of-the-art OSINT tools and techniques. Technical users will want a basic understanding of the Linux command line in order to follow the examples. But a person with no Linux or programming experience can still gain a lot from this book through the commentaries.
This book’s unique digital investigation proposition is a combination of story-telling, tutorials, and case studies. The book explores digital investigation from multiple angles:
- Through the eyes of the author who has several years of experience in the subject.
- Through the mind of the hacker who collects massive amounts of data from multiple online sources to identify targets as well as ways to hit the targets.
- Through the eyes of industry leaders.
This book is ideal for:
Investigation professionals, forensic analysts, and CISO/CIO and other executives wanting to understand the mindset of a hacker and how seemingly harmless information can be used to target their organization.
Security analysts, forensic investigators, and SOC teams looking for new approaches on digital investigations from the perspective of collecting and parsing publicly available information.
CISOs and defense teams will find this book useful because it takes the perspective of infiltrating an organization from the mindset of a hacker. The commentary provided by outside experts will also provide them with ideas to further protect their organization’s data.
Table of Contents
Prologue xxv
Chapter 1 Getting Started 1
Why This Book is Different 2
What You Will and Won’t Find in This Book 2
Getting to Know Your Fellow Experts 3
A Note on Cryptocurrencies 4
What You Need to Know 4
Paid Tools and Historical Data 5
What about Maltego? 5
Prerequisites 5
Know How to Use and Configure Linux 5
Get Your API Keys in Order 6
Important Resources 6
OSINT Framework 6
OSINT.link 6
IntelTechniques 7
Termbin 8
Hunchly 9
Wordlists and Generators 9
SecLists 9
Cewl 10
Crunch 10
Proxies 10
Storm Proxies (Auto-Rotating) 10
Cryptocurrencies 101 11
How Do Cryptocurrencies Work? 12
Blockchain Explorers 13
Following the Money 15
Identifying Exchanges and Traders 17
Summary 18
Chapter 2 Investigations and Threat Actors 19
The Path of an Investigator 19
Go Big or Go Home 20
The Breach That Never Happened 21
What Would You Do? 22
Moral Gray Areas 24
Different Investigative Paths 25
Investigating Cyber Criminals 26
The Beginning of the Hunt (for TDO) 27
The Dark Overlord 27
List of Victims 28
A Brief Overview 29
Communication Style 30
Group Structure and Members 30
Cyper 31
Arnie 32
Cr00k (Ping) 35
NSA (Peace of Mind) 36
The Dark Overlord 38
Summary 41
Part I Network Exploration 43
Chapter 3 Manual Network Exploration 45
Chapter Targets: Pepsi.com and Cyper.org 46
Asset Discovery 46
ARIN Search 47
Search Engine Dorks 48
DNSDumpster 49
Hacker Target 52
Shodan 53
Censys (Subdomain Finder) 56
Censys Subdomain Finder 56
Fierce 57
Sublist3r 58
Enumall 59
Results 60
Phishing Domains and Typosquatting 61
Summary 64
Chapter 4 Looking for Network Activity (Advanced NMAP Techniques) 67
Getting Started 67
Preparing a List of Active Hosts 68
Full Port Scans Using Different Scan Types 68
TCP Window Scan 70
Working against Firewalls and IDS 70
Using Reason Response 71
Identifying Live Servers 71
Firewall Evasion 73
Distributed Scanning with Proxies and TOR 73
Fragmented Packets/MTU 74
Service Detection Trick 74
Low and Slow 76
Bad Checksums, Decoy, and Random Data 76
Firewalking 79
Comparing Results 79
Styling NMAP Reports 81
Summary 82
Chapter 5 Automated Tools for Network Discovery 83
SpiderFoot 84
SpiderFoot HX (Premium) 91
Intrigue.io 95
Entities Tab 96
Analyzing uberpeople.net 99
Analyzing the Results 104
Exporting Your Results 105
Recon-NG 107
Searching for Modules 111
Using Modules 111
Looking for Ports with Shodan 115
Summary 116
Part II Web Exploration 119
Chapter 6 Website Information Gathering 121
BuiltWith 121
Finding Common Sites Using Google Analytics Tracker 123
IP History and Related Sites 124
Webapp Information Gatherer (WIG) 124
CMSMap 129
Running a Single Site Scan 130
Scanning Multiple Sites in Batch Mode 130
Detecting Vulnerabilities 131
WPScan 132
Dealing with WAFs/WordPress Not Detected 136
Summary 141
Chapter 7 Directory Hunting 143
Dirhunt 143
Wfuzz 146
Photon 149
Crawling a Website 151
Intrigue.io 152
Summary 157
Chapter 8 Search Engine Dorks 159
Essential Search Dorks 160
The Minus Sign 160
Using Quotes 160
The site: Operator 161
The intitle: Operator 161
The allintitle: Operator 162
The fi letype: Operator 162
The inurl: Operator 163
The cache: Operator 165
The allinurl: Operator 165
The fi lename: Operator 165
The intext: Operator 165
The Power of the Dork 166
Don’t Forget about Bing and Yahoo! 169
Automated Dorking Tools 169
Inurlbr 169
Using Inurlbr 171
Summary 173
Chapter 9 WHOIS 175
WHOIS 175
Uses for WHOIS Data 176
Historical WHOIS 177
Searching for Similar Domains 177
Namedroppers.com 177
Searching for Multiple Keywords 179
Advanced Searches 181
Looking for Threat Actors 182
Whoisology 183
Advanced Domain Searching 187
Worth the Money? Absolutely 188
DomainTools 188
Domain Search 188
Bulk WHOIS 189
Reverse IP Lookup 189
WHOIS Records on Steroids 190
WHOIS History 192
The Power of Screenshots 193
Digging into WHOIS History 193
Looking for Changes in Ownership 194
Reverse WHOIS 196
Cross-Checking All Information 197
Summary 199
Chapter 10 Certificate Transparency and Internet Archives 201
Certificate Transparency 201
What Does Any of This Have to Do with Digital Investigations? 202
Scouting with CTFR 202
Crt.sh 204
CT in Action: Side-stepping Cloudflare 204
Testing More Targets 208
CloudFlair (Script) and Censys 209
How Does It Work? 210
Wayback Machine and Search Engine Archives 211
Search Engine Caches 212
CachedView.com 214
Wayback Machine Scraper 214
Enum Wayback 215
Scraping Wayback with Photon 216
Archive.org Site Search URLs 217
Wayback Site Digest: A List of Every Site URL Cached by Wayback 219
Summary 220
Chapter 11 Iris by DomainTools 221
The Basics of Iris 221
Guided Pivots 223
Configuring Your Settings 223
Historical Search Setting 224
Pivootttt!!! 225
Pivoting on SSL Certificate Hashes 227
Keeping Notes 228
WHOIS History 230
Screenshot History 232
Hosting History 232
Bringing It All Together 234
A Major Find 240
Summary 241
Part III Digging for Gold 243
Chapter 12 Document Metadata 245
Exiftool 246
Metagoofil 248
Recon-NG Metadata Modules 250
Metacrawler 250
Interesting_Files Module 252
Pushpin Geolocation Modules 254
Intrigue.io 257
FOCA 261
Starting a Project 262
Extracting Metadata 263
Summary 266
Chapter 13 Interesting Places to Look 267
TheHarvester 268
Running a Scan 269
Paste Sites 273
Psbdmp.ws 273
Forums 274
Investigating Forum History (and TDO) 275
Following Breadcrumbs 276
Tracing Cyper’s Identity 278
Code Repositories 280
SearchCode.com 281
Searching for Code 282
False Negatives 283
Gitrob 284
Git Commit Logs 287
Wiki Sites 288
Wikipedia 289
Summary 292
Chapter 14 Publicly Accessible Data Storage 293
The Exactis Leak and Shodan 294
Data Attribution 295
Shodan’s Command-Line Options 296
Querying Historical Data 296
CloudStorageFinder 298
Amazon S3 299
Digital Ocean Spaces 300
NoSQL Databases 301
MongoDB 302
Robot 3T 302
Mongo Command-Line Tools 305
Elasticsearch 308
Querying Elasticsearch 308
Dumping Elasticsearch Data 311
NoScrape 311
MongoDB 313
Elasticsearch 314
Scan 314
Search 315
Dump 317
MatchDump 317
Cassandra 318
Amazon S3 320
Using Your Own S3 Credentials 320
Summary 321
Part IV People Hunting 323
Chapter 15 Researching People, Images, and Locations 325
PIPL 326
Searching for People 327
Public Records and Background Checks 330
Ancestry.com 331
Threat Actors Have Dads, Too 332
Criminal Record Searches 332
Image Searching 333
Google Images 334
Searching for Gold 335
Following the Trail 335
TinEye 336
EagleEye 340
Searching for Images 340
Cree.py and Geolocation 343
Getting Started 343
IP Address Tracking 346
Summary 347
Chapter 16 Searching Social Media 349
OSINT.rest 350
Another Test Subject 355
Twitter 357
SocialLinks: For Maltego Users 358
Skiptracer 361
Running a Search 361
Searching for an Email Address 361
Searching for a Phone Number 364
Searching Usernames 366
One More Username Search 368
Userrecon 370
Reddit Investigator 372
A Critical “Peace” of the TDO Investigation 374
Summary 375
Chapter 17 Profile Tracking and Password Reset Clues 377
Where to Start (with TDO)? 377
Building a Profile Matrix 378
Starting a Search with Forums 379
Ban Lists 381
Social Engineering 381
SE’ing Threat Actors: The “Argon” Story 383
Everyone Gets SE’d - a Lesson Learned 387
The End of TDO and the KickAss Forum 388
Using Password Reset Clues 390
Starting Your Verification Sheet 391
Gmail 391
Facebook 393
PayPal 394
Twitter 397
Microsoft 399
Instagram 400
Using jQuery Website Responses 400
ICQ 403
Summary 405
Chapter 18 Passwords, Dumps, and Data Viper 407
Using Passwords 408
Completing F3ttywap’s Profile Matrix 409
An Important Wrong Turn 412
Acquiring Your Data 413
Data Quality and Collections 1-5 413
Always Manually Verify the Data 415
Where to Find Quality Data 420
Data Viper 420
Forums: The Missing Link 421
Identifying the Real “Cr00k” 422
Tracking Cr00k’s Forum Movements 423
Timeline Analysis 423
The Eureka Moment 427
Vanity over OPSEC, Every Time 429
Why This Connection is Significant 429
Starting Small: Data Viper 1.0 430
Summary 431
Chapter 19 Interacting with Threat Actors 433
Drawing Them Out of the Shadows 433
Who is WhitePacket? 434
The Bev Robb Connection 435
Stradinatras 436
Obfuscation and TDO 437
Who is Bill? 439
So Who Exactly is Bill? 440
YoungBugsThug 440
How Did I Know It Was Chris? 441
A Connection to Mirai Botnet? 442
Why Was This Discovery So Earth-Shattering? 444
Question Everything! 445
Establishing a Flow of Information 446
Leveraging Hacker Drama 447
Was Any of That Real? 448
Looking for Other Clues 449
Bringing It Back to TDO 450
Resolving One Final Question 451
Withdrawing Bitcoin 451
Summary 452
Chapter 20 Cutting through the Disinformation of a 10-Million-Dollar Hack 453
GnosticPlayers 454
Sites Hacked by GnosticPlayers 456
Gnostic’s Hacking Techniques 457
GnosticPlayers’ Posts 459
GnosticPlayers2 Emerges 461
A Mysterious Third Member 462
NSFW/Photon 463
The Gloves Come Off 464
Making Contact 465
Gabriel/Bildstein aka Kuroi’sh 465
Contacting His Friends 467
Weeding through Disinformation 468
Verifying with Wayback 468
Bringing It All Together 469
Data Viper 469
Trust but Verify 472
Domain Tools’ Iris 474
Verifying with a Second Data Source 475
The End of the Line 476
What Really Happened? 476
Outofreach 476
Kuroi’sh Magically Appears 477
What I Learned from Watching Lost 477
Who Hacked GateHub? 478
Unraveling the Lie 479
Was Gabriel Involved? My Theory 479
Gabriel is Nclay: An Alternate Theory 479
All roads lead back to NSFW 480
Summary 481
Epilogue 483
Index 487
