In Causal Artificial Intelligence: The Next Step in Effective, Efficient, and Practical AI, a team of dedicated tech executives delivers a business-focused approach based on a deep and engaging exploration of the models and data used in causal AI. The book’s discussions include both accessible and understandable technical detail and business context and concepts that frame causal AI in familiar business settings.
Useful for both data scientists and business-side professionals, the book offers: - Clear and compelling descriptions of the concept of causality and how it can benefit your organization - Detailed use cases and examples that vividly demonstrate the value of causality for solving business problems - Useful strategies for deciding when to use correlation-based approaches and when to use causal inference
An enlightening and easy-to-understand treatment of an essential business topic, Causal Artificial Intelligence is a must-read for data scientists, subject matter experts, and business leaders seeking to familiarize themselves with a rapidly growing area of AI application and research.
Table of Contents
Foreword xix
Preface xxiii
Introduction xxix
Chapter 1 Setting the Stage for Causal AI 1
Why Causality Is a Game Changer 2
Causal AI in Perspective with Analytics 7
Analytical Sophistication Model 8
Analytics Enablers 10
Analytics 10
Advanced Analytics 11
Scope of Services to Support Causal AI 11
The Value of the Hybrid Team 13
The Promise of AI 14
Understanding the Core Concepts of Causal AI 15
Explainability and Bias Detection 15
Explainability 17
Detecting Bias in a Model 17
Directed Acyclic Graphs 18
Structural Causal Model 19
Observed and Unobserved Variables 20
Counterfactuals 21
Confounders 21
Colliders 22
Front- Door and Backdoor Paths 23
Correlation 24
Causal Libraries and Tools 25
Propensity Score 25
Augmented Intelligence and Causal AI 26
Summary 27
Note 27
Chapter 2 Understanding the Value of Causal AI 29
Defining Causal AI 30
The Origins of Causal AI 33
Why Causality? 34
Expressing Relationships 37
The Ladder of Causation 38
Rung 1: Association, or Passive Observation 40
Rung 2: Intervention, or Taking Action 40
Rung 3: Counterfactuals, or Imagining What If 42
Why Causal AI Is the Next Generation of AI 43
Deep Learning and Neural Networks 43
Neural Networks 44
Establishing Ground Truth 45
The Business Imperative of a Causal Model 46
The Importance of Augmented Intelligence 51
The Importance of Data, Visualization, and Frameworks 52
Getting the Appropriate Data 52
Applying Data and Model Visualization 55
Applying Frameworks After Creating a Model 56
Getting Started with Causal AI 57
Summary 58
Notes 59
Chapter 3 Elements of Causal AI 61
Conceptual Models 62
Correlation vs. Causal Models 63
Correlation- Based AI 63
Causal AI 63
Understanding the Relationship Between Correlation and Causality 64
Process Models 66
Correlation- Based AI Process Model 67
Causal- Based AI Process Model 69
Collaboration Between Business and Analytics Professionals 72
The Fundamental Building Blocks of Causal AI Models 75
The Relations Between DAGs and SCMs 76
Explaining DAGs 76
Causal Notation: The Language of DAGs 78
Operationalizing a DAG with an SCM 79
The Elements of Visual Modeling 81
Nodes 83
Variables 83
Endogenous and Exogenous Variables 83
Observed and Unobserved Variables 84
Paths/Relationships 84
Weights 86
Summary 88
Notes 89
Chapter 4 Creating Practical Causal AI Models and Systems 91
Understanding Complex Models 92
Causal Modeling Process: Part 1 94
Step 1: What Are the Intended Outcomes? 95
Step 2: What Are the Proposed Interventions? 97
Step 3: What Are the Confounding Factors? 99
Step 4: What Are the Factors Creating the Effects and Changes? 102
Common/Universal Effects in a Causal Model 102
Refined Effects in a Causal Model 103
Step 5: Creating a Directed Acyclic Graph 105
Step 6: Paths and Relationships 105
Types of Paths 106
Path Connecting an Unobserved Variable 107
Front- Door Paths 108
Backdoor Paths 108
Modeling for Simplicity to Understand Complexity 109
Step 7: Data Acquisition 110
Causal- Based Approach: Part 2 112
Step 8: Data Integration 113
Step 9: Model Modification 114
Step 10: Data Transformation 115
Step 11: Preparing for Deployment in Business 118
Summary 121
Notes 122
Chapter 5 Creating a Model with a Hybrid Team 125
The Hybrid Team 126
Why a Hybrid Team? 127
The Benefits of a Hybrid Team 128
Establishing the Hybrid Team as a Center of Excellence 129
How Teams Collaborate 131
But Why? 132
Defining Roles 134
Leaders and Business Strategists 137
Subject- Matter Experts 138
Data Experts 140
Software Developers 142
Business Process Analysts 143
Information Technology Expertise 143
Project Manager(s) 144
The Basics Steps for a Hybrid Team Project 145
An Overview of Model Creation 146
It Depends on Your Destination 150
Understanding the Root Cause of a Problem 151
Understanding What Happened and Why 153
The Importance of the Iterative Process 154
Summary 155
Chapter 6 Explainability, Bias Detection, and AI Responsibility in Causal AI 157
Explainability 158
The Ramifications of the Lack of Explainability 159
What Is Explainable AI in Causal AI Models? 161
Black Boxes 162
Internal Workings of Black-Box Models 162
Deep Learning at the Heart of Black Boxes 163
Is Code Understandable? 163
The Value of White-Box Models 166
Understanding Causal AI Code 167
Techniques for Achieving Explainability 169
Challenges of Complex Causal Models 169
Methods for Understanding and Explaining Complex Causal AI Models 171
The Importance of the SHAP Explainability Method 172
Detecting Bias and Ensuring Responsible AI 175
Bias in Causal AI Systems 176
Responsible AI: Trust and Fairness 178
How Causal AI Addresses Bias Detection 180
Tools for Assessing Fairness and Bias 182
The Human Factor in Bias Detection and Responsible AI 183
Summary 184
Note 184
Chapter 7 Tools, Practices, and Techniques to Enable Causal AI 185
The Causal AI Pipeline 187
Define Business Objectives 190
Model Development 193
Data Identification and Collection 195
Data Privacy, Governance, and Security 197
Synthetic Data 198
Model Validation 199
Deployment/Production 201
Monitor and Evaluate 203
Update and Iterate 205
Continuous Learning 208
The Importance of Synthetic Data 210
Why Create Synthetic Data? 210
Overcoming Data Limitations 211
Enhancing Data Privacy and Security 211
Model Validation and Testing 211
Expanding the Range of Possible Scenarios 212
Reducing the Cost of Data Collection 212
Improving Data Imbalance 213
Encouraging Collaboration and Openness 213
Streamlining Data Preprocessing 213
Supporting Counterfactual Analysis 213
Fostering Innovation and Experimentation 214
Creating Synthetic Data 214
Generative Models 214
Agent-Based Modeling 215
Data Augmentation 215
Data Synthesis Tools and Platforms 215
Conditional Synthetic Data Generation 216
Synthetic Data from Text 216
The Limitations of Synthetic Data 217
Current State of Tools and Software in Causal AI 218
The Role of Open Source in Causal AI 218
Commercial Causal AI Software 221
CausaLens 221
Geminos Software 223
Summary 223
Chapter 8 Causal AI in Action 225
Enterprise Marketing in a Business- to- Consumer Scenario 226
DDCo Marketing Causal Model: Annual Pricing Review and Update Cycle 228
Incorporating Internal and External Factors in the Model and DAG 230
Easily Enabling Iterating 231
End-User-Driven Exploration 232
Bench Testing 234
DDCo Marketing Causal Model: Semiannual Product Planning Cycle 236
Always Consider Model Reuse 237
Give and Take in Building a New Model 239
Typical Model and Process Operation: Iterating 239
Keeping the Process/Model Scope Manageable and Understandable 240
Moving from Strategy to Building and Implementing Causal AI Solutions 241
Agriculture: Enhancing Crop Yield 242
Key Causal Variables 244
Creating the DAG 246
Moving from the DAG to Implementing the Causal AI Model 247
Commercial Real Estate: Valuing Warehouse Space 250
Key Causal Variables 251
Implementing the Causal AI Model 253
Video Streaming: Enhancing Content Recommendations 254
Key Causal Variables 255
Implementing the Causal AI Model 256
Healthcare: Reducing Infant Mortality 258
Key Causal Variables 259
Implementing the Causal AI Model 261
Retail: Providing Executives Actionable Information 263
Key Causal Variables 264
Implementing the Causal Model 265
Summary 267
Chapter 9 The Future of Causal AI 271
Where We Stand Today 271
Foundations of Causal AI 273
The Causal AI Journey 274
Causal AI Today 274
What’s Next for Causal AI 276
Integrating Causal AI and Traditional AI 278
The Imperative for Managing Data 279
Ensembles of Data 279
Generative AI Is Emerging as a Game Changer for Causal AI 281
The Future of Causal Discovery 282
The Emergence of Causal AI Reinforcement Learning Will Accelerate Model Training 284
Causal AI as a Common Language Between Business Leaders and Data Scientists 284
The Emergence of Probabilistic Programming Languages 286
The Predictable Model Evolution Cycle 286
The Emergence of the Digital Twin 287
Improving the Ability to Understand Ground Truth 289
The Development of More Sophisticated DAGs 289
Visualizing Complex Relationships in the DAGs 290
The Merging of Causal and Traditional AI Models 291
The Future of Explainability 291
The Evolution of Responsible AI 292
Advances in Data Security and Privacy 293
Integration Will Be Between Models and Business Applications 294
Summary 295
Glossary 299
Appendix 313
Selected Resources 329
Acknowledgments 331
About the authors 335
About the contributor 339
Index 341
