Streamlining Your Research Laboratory with Python [Kõva köide]

Preface xv

1 Introduction 1

1.1 Python Implementations 1

1.2 Installing the Python Toolkit 2

1.3 Python 3 vs. Python 2 2

1.4 Python Package Index 3

1.5 Programming Editors 5

1.6 Notebook Editors 5

1.7 Using the Jupyter Notebook Interface 7

1.8 JupyterLite 8

1.9 Things Change 9

1.10 Key Takeaways 9

2 Language Basics 11

2.1 Python Interactive Console 11

2.2 Data Types 11

2.3 Variables and Literals 14

2.4 Strings 15

2.4.1 Simple Strings 15

2.4.2 Multi-Line Strings 16

2.4.3 Escape Characters in a String 16

2.4.4 Raw Strings 18

2.4.5 Formatted Strings 18

2.4.6 Strings as Objects 19

2.4.7 Characters and Encodings 21

2.5 Expressions Using Operators 22

2.5.1 Arithmetic Operators 22

2.5.2 Assignment Operators 24

2.5.3 Comparison Operators 25

2.5.4 Boolean Operators 26

2.5.5 Chaining Comparisons 28

2.5.6 Comparing Floating-Point Numbers 29

2.6 Functions and How to Use Them 29

2.6.1 Invoking Functions 30

2.6.2 Built-in Functions 31

2.6.3 The math Module for Additional Mathematical Functions 31

2.6.4 The random Module for Pseudo-Random Number Generation 33

2.6.5 The time and datetime Modules for Handling Dates and Times 36

2.6.6 The sys Module for System Interactions 39

2.6.7 Scope and Namespace 40

2.7 Your First Python Program 41

2.8 Key Takeaways 42

3 Data Structures 45

3.1 Lists 45

3.1.1 Introducing Lists 45

3.1.2 Global Functions That Operate on Lists 46

3.1.3 Accessing List Elements 46

3.1.4 Slicing Lists 47

3.1.5 Lists Operators 49

3.1.6 Lists as Objects 51

3.2 Tuples 55

3.2.1 Introducing Tuples 55

3.3 Dictionaries 56

3.3.1 Introducing Dictionaries 56

3.3.2 Global Functions That Operate on Dictionaries 57

3.3.3 Accessing Dictionary Items 57

3.3.4 Dictionary Operators 58

3.3.5 Dictionaries as Objects 59

3.4 Sets 61

3.4.1 Introducing Sets 61

3.4.2 Global Functions That Operate on Sets 62

3.4.3 Accessing Set Elements 62

3.4.4 Set Operators 62

3.4.5 Sets as Objects 64

3.5 Destructuring Assignment 65

3.6 Key Takeaways 66

4 Controlling the Flow of a Program 67

4.1 Conditional Execution 67

4.1.1 If-Statements 67

4.1.2 If-Else Statements 68

4.1.3 If-Elif-Else Statements 70

4.1.4 If-Statement Strategies 71

4.1.5 Truthy and Falsy Values 74

4.1.6 Conditional Expressions 75

4.2 Repeated Execution 76

4.2.1 While-Statements 76

4.2.2 For-Statements 81

4.2.3 For-Statements with Range 83

4.2.4 Break and Continue 84

4.2.5 Comprehensions 85

4.3 Key Takeaways 87

5 Custom Functions and Exceptions 89

5.1 Defining Custom Functions 89

5.2 Arguments and Parameters 94

5.3 Names and Scope 97

5.3.1 Local vs. Global 97

5.3.2 Built-in and Nonlocal Scope 99

5.4 Scope vs. Namespace 100

5.5 Organizing Your Code with Modules 102

5.6 Decorators 104

5.7 How Things Go Wrong 106

5.8 Python Exceptions 106

5.9 Handling Exceptions 107

5.10 Raising Your Own Exceptions 110

5.11 Key Takeaways 112

6 Regular Expressions 115

6.1 Matching Literal Text 115

6.2 Alternation 116

6.3 Defining and Matching Character Classes 116

6.4 Metaclasses 117

6.5 Pattern Sequences 117

6.6 Repeating Patterns with Quantifiers 118

6.7 Anchors 119

6.8 Capturing Groups 120

6.9 Regular Expressions in Python 122

6.10 Project A Formula Mass Calculator 123

6.11 Key Takeaways 130

7 Working with Data 131

7.1 A File System Primer 131

7.2 Text Files 133

7.3 Reading and Writing Text Files 134

7.4 Working with Comma-Separated Values (CSV) Files 139

7.5 The csv Module 144

7.6 Reading and Writing Excel Spreadsheet 145

7.6.1 openpyxl Workbook Object 146

7.6.2 openpyxl Worksheet Object 147

7.6.3 openpyxl Cell Object 148

7.7 Project Generate a Random Sample Layout in a Spreadsheet 148

7.8 Project Forecast Monthly Sample Processing 150

7.9 Managing the File System 154

7.9.1 The Path Object 155

7.9.2 Path Properties 155

7.9.3 Path Attributes 156

7.9.4 Operating On a Path 157

7.9.5 Combining Paths 158

7.9.6 The shutil Module for High-Level File Operations 159

7.10 Walking a File System Tree 159

7.11 Project Find Duplicate Files 161

7.12 Working with Zip Files 163

7.12.1 ZipFile Object 163

7.12.2 zipfile Path Object 163

7.12.3 Creating Zip Archives 164

7.13 Working with Standard Data Formats 165

7.13.1 JSON JavaScript Object Notation 165

7.13.2 json Python Module 166

7.13.3 XML Extensible Markup Language 168

7.13.4 Python XML modules 170

7.13.5 Other Standard Data Formats 176

7.14 Key Takeaways 176

8 Web Resources 179

8.1 TCP/IP Networks What You Need to Know 179

8.1.1 Internet Protocol 180

8.1.2 Transmission Control Protocol 180

8.1.3 Connections and Ports 180

8.1.4 Application Layer Protocols 181

8.1.5 IPv4 vs. IPv6 Addresses 181

8.1.6 Proxy Servers 181

8.2 Introduction to Hypertext Transfer Protocol 182

8.2.1 The Uniform Resource Locator 183

8.2.2 Anatomy of an HTTP Request 184

8.2.3 Anatomy of an HTTP Response 186

8.3 Web Services and the Python Requests Module 187

8.3.1 HTTP GET Requests and the Response Object 188

8.3.2 HTTP POST Requests 189

8.3.3 Binary Responses 190

8.3.4 Customizing the Request Object 193

8.3.5 Verifying Certificates and Encryption 193

8.3.6 Other requests Module Options 194

8.4 Project Print Weather Forecast for a Location 195

8.4.1 National Weather Service API Web Service 195

8.4.2 Getting Forecast URL from Geolocation 197

8.4.3 Loading and Processing Forecast Data 197

8.4.4 Completed Program to Generate Temperature Forecast 198

8.5 Project Scraping HTML Page Content 201

8.6 Key Takeaways 206

9 Data Analysis and Visualization 209

9.1 JupyterLab 209

9.2 Scientific Plotting with Matplotlib 211

9.2.1 The pyplot Submodule 212

9.2.2 The pyplot.plot() function 212

9.2.3 Customizing a Plot 213

9.2.4 Multiple Curves on a Single Plot 214

9.2.5 Additional Plot Types 214

9.2.6 Multiple Axes on a Single Figure 214

9.2.7 Other Useful Functions 216

9.2.8 Project Plotting Weather Forecast 216

9.2.9 Project A Custom Microplate Heat Map 219

9.2.10 Other Scientific Plotting Libraries 222

9.3 NumPy Numerical Python 223

9.3.1 Creating ndarray Objects 223

9.3.2 Working with ndarray Objects 223

9.3.3 Accessing and Updating ndarray Elements 224

9.3.4 Broadcasting 226

9.4 pandas DataFrame 226

9.4.1 Creating and Inspecting DataFrames 227

9.4.2 Filtering DataFrames 228

9.4.3 Project A Screening Experiment 230

9.5 SciPy A Library for Mathematics, Science, and Engineering 239

9.5.1 Descriptive Statistics with SciPy 239

9.5.2 Hypothesis Testing 239

9.5.3 Project Running Hypothesis Tests on Two Samples 240

9.5.4 Project Comparing Liquid Handler Syringe Performance 243

9.5.5 Linear Regression 245

9.5.6 Fitting Nonlinear Models to Data 246

9.5.7 Project Four-Parameter Logistic Regression 248

9.6 Key Takeaways 253

10 Report Generation 255

10.1 BytesIO Object 255

10.2 Generating Reports in Microsoft Word 257

10.2.1 Document Object 257

10.2.2 Paragraph Object 258

10.2.3 Run Object 259

10.2.4 Picture and InlineShape Objects 260

10.2.5 Table Object 261

10.2.6 Project Generate a Complete Word Report 263

10.3 Generating Microsoft PowerPoint Presentations 266

10.3.1 Presentation Object 266

10.3.2 Slide Objects 267

10.3.3 SlideShapes Object 268

10.3.4 Length Objects 269

10.3.5 Table Object 269

10.3.6 Project Generate a PowerPoint Document with Figures and Tables 272

10.4 Generating PDF File Reports 275

10.4.1 ReportLab PDF Generation Process 275

10.4.2 Creating a Canvas Object 276

10.4.3 Setting Canvas Styles 276

10.4.4 Managing Text Blocks with PDFTextObjects 278

10.4.5 Canvas State Stack 281

10.4.6 Drawing Images 281

10.4.7 PLATYPUS for Page Layout 283

10.4.8 Project Generate a Complete PDF Report 283

10.5 Sending Email Programmatically 287

10.5.1 Simple Mail Transfer Protocol 288

10.5.2 SMTP Mail Server 288

10.5.3 Send a Simple Email Message 288

10.5.4 Sending Email Messages over a Secure Connection 291

10.5.5 Building an Email Message with Attachments 292

10.6 Serving Results with an HTTP Server 294

10.7 Key Takeaways 296

11 Control and Automation 297

11.1 Concurrency in Python 297

11.2 Asynchronous Execution 299

11.3 Concurrent Programs with AsyncIO 300

11.4 Asynchronous Instrument Control and Coordination 304

11.4.1 Project Integrated Laboratory System Control and Coordination 304

11.5 Communicating over a Serial Port 311

11.5.1 Reading Barcodes from a Serial Port 312

11.5.2 Project Scanning Sample Tasks into a Running Controller 318

11.6 Execute Remote Commands over HTTP 319

11.6.1 A Basic HTTP Server with aiohttp 320

11.6.2 Routing an HTTP Request to a Custom Python Function 321

11.7 Persistent Network Connections using a WebSocket 325

11.7.1 A User Interface for Asynchronous Networked Programs 326

11.7.2 WebSocketResponse and FileResponse Objects 326

11.7.3 Project A Browser-Based WebSocket Message Broadcaster 327

11.7.4 Project A Browser User Interface to Schedule Samples for Analysis
335

11.8 Responding to File System Changes 338

11.8.1 Watching a Directory for Changes with watchfiles 338

11.8.2 File System Monitoring Options 340

11.9 Executing Tasks on a Schedule 341

11.9.1 sched Module 342

11.9.2 Project Taking and Sending Images on a Schedule 344

11.10 Key Takeaways 348

Postface 351

References 353

Appendix A: ASCII American Standard Code for Information Interchange 357

Index 359

Mark F. Russo, PhD is currently on the faculty in the Department of Computer Science at The College of New Jersey. Previously, he had a multi-decade professional career in biotech and pharma with a focus on scientific computing, automation, and scientific data.

William Neil is currently at Bristol Myers Squibb and has been working in the pharmaceutical industry since 1995.