Website Search
Find information on spaces, staff, and services.
Catalog Search
Books
Big data for chimps
- Author / Creator
- Available as
- Online
- Summary
-
Finding patterns in massive event streams can be difficult, but learning how to find them doesn’t have to be. This unique hands-on guide shows you how to solve this and many other problems in large...
Finding patterns in massive event streams can be difficult, but learning how to find them doesn’t have to be. This unique hands-on guide shows you how to solve this and many other problems in large-scale data processing with simple, fun, and elegant tools that leverage Apache Hadoop. You’ll gain a practical, actionable view of big data by working with real data and real problems. Perfect for beginners, this book’s approach will also appeal to experienced practitioners who want to brush up on their skills. Part I explains how Hadoop and MapReduce work, while Part II covers many analytic patterns you can use to process any data. As you work through several exercises, you’ll also learn how to use Apache Pig to process data.
Details
- Creator
- Philip Kromer & Russell Jurney
- Format
- Books
- Language
- English
- Contributors
- Publication
-
- First edition
- Sebastopol, CA : O'Reilly, 2015
- Physical Details
-
- 1 online resource (219 p.)
- text file
- ISBNs
- 9781491923900, 1491923903, 9781491923931, 1491923938, 9781491923924, 149192392X, 1491923946
- OCLC
- ocn924210357
- Includes index.
- English
- Copyright; Table of Contents; Preface; What This Book Covers; Who This Book Is For; Who This Book Is Not For; What This Book Does Not Cover; Theory: Chimpanzee and Elephant; Practice: Hadoop; Example Code; A Note on Python and MrJob; Helpful Reading; Feedback; Conventions Used in This Book; Using Code Examples; Safari® Books Online; How to Contact Us; Part I. Introduction: Theory and Tools; Chapter 1. Hadoop Basics; Chimpanzee and Elephant Start a Business; Map-Only Jobs: Process Records Individually; Pig Latin Map-Only Job; Setting Up a Docker Hadoop Cluster; Run the Job; Wrapping Up
- Chapter 2. MapReduceChimpanzee and Elephant Save Christmas; Trouble in Toyland; Chimpanzees Process Letters into Labeled Toy Forms; Pygmy Elephants Carry Each Toy Form to the Appropriate Workbench; Example: Reindeer Games; UFO Data; Group the UFO Sightings by Reporting Delay; Mapper; Reducer; Plot the Data; Reindeer Conclusion; Hadoop Versus Traditional Databases; The MapReduce Haiku; Map Phase, in Light Detail; Group-Sort Phase, in Light Detail; Reduce Phase, in Light Detail; Wrapping Up; Chapter 3. A Quick Look into Baseball; The Data; Acronyms and Terminology; The Rules and Goals
- Performance MetricsWrapping Up; Chapter 4. Introduction to Pig; Pig Helps Hadoop Work with Tables, Not Records; Wikipedia Visitor Counts; Fundamental Data Operations; Control Operations; Pipelinable Operations; Structural Operations; LOAD Locates and Describes Your Data; Simple Types; Complex Type 1, Tuples: Fixed-Length Sequence of Typed Fields; Complex Type 2, Bags: Unbounded Collection of Tuples; Defining the Schema of a Transformed Record; STORE Writes Data to Disk; Development Aid Commands; DESCRIBE; DUMP; SAMPLE; ILLUSTRATE; EXPLAIN; Pig Functions; Piggybank; Apache DataFu; Wrapping Up
- Part II. Tactics: Analytic PatternsChapter 5. Map-Only Operations; Pattern in Use; Eliminating Data; Selecting Records That Satisfy a Condition: FILTER and Friends; Selecting Records That Satisfy Multiple Conditions; Selecting or Rejecting Records with a null Value; Selecting Records That Match a Regular Expression (MATCHES); Matching Records Against a Fixed List of Lookup Values; Project Only Chosen Columns by Name; Using a FOREACH to Select, Rename, and Reorder fields; Extracting a Random Sample of Records; Extracting a Consistent Sample of Records by Key
- Sampling Carelessly by Only Loading Some part- FilesSelecting a Fixed Number of Records with LIMIT; Other Data Elimination Patterns; Transforming Records; Transforming Records Individually Using FOREACH; A Nested FOREACH Allows Intermediate Expressions; Formatting a String According to a Template; Assembling Literals with Complex Types; Manipulating the Type of a Field; Ints and Floats and Rounding, Oh My!; Calling a User-Defined Function from an External Package; Operations That Break One Table into Many; Directing Data Conditionally into Multiple Dataflows (SPLIT)
- Operations That Treat the Union of Several Tables as One