Đang tải... (xem toàn văn)
Thuyết trình big data
Big Data GVGD: TS Nguyễn Đức Thái NHÓM Memory storage… Computer Memory: 640K Ought to be Enough for Anyone How much data? billion people Google processes 100 PB/day; million servers Facebook has 300 PB + 500 TB/day; 35% of world’s photos YouTube 1000 PB video storage; billion views/day Twitter processes 124 billion tweets/year SMS messages – 6.1T per year US Cell Calls – 2.2T minutes per year US Credit cards - 1.4B Cards; 20B transactions/year Contents Big Data Overview Big Data Technology Today SQL vs NoSQL Big Data Security Big data trends Demo with MongoDB & Ref docs Big Data Overview (tt) “Big data is not a single technology but a combination of old and new tech-nologies that helps companies gain actionable insight” (“Big Data For DummiesPublished by John Wiley & Sons, Inc ” book reference) Big Data Overview (tt) Characteristics of Big Data Sources of Big Data Social Media Website ERP Network Switches RFID Examining Big Data Types Structured Data Structured Data(…) Computer- or machine-generated: Machine-generated data generally refers to data that is created by a machine without human intervention (Sensor data, Web log data, Point-ofsale data, Financial data…) Human-generated: This is data that humans, in interaction with computers, supply (Input data, Clickstream data, Gaming-related data…) 2.Big Data Technology Today(tt) Open-source software framework from Apache Hadoop Google MapReduce GFS (Google File System) HDFS Map/Reduce SQL vs NoSQL File SQL DBMS Data storage NoSQL SQL vs NoSQL (…) A relational database is a set of tables containing data fitted into predefined categories Each table contains one or more data categories in columns Each row contains a unique instance of data for the categories defined by the columns SQL vs NoSQL (…) Key-value stores As the name implies, a key-value store is a system that stores values indexed for retrieval by keys Some of the market leaders: Riak Amazon Dynamo Voldermort SQL vs NoSQL (…) Column-oriented databases columnoriented databases contain one extendable column of closely related data Some of the market leaders: HBase Cassandra SQL vs NoSQL (…) Document-based stores These databases store and organize data as collections of documents, rather than as structured tables with uniform sized fields for each record Some of the market leaders: MongoDB CouchDB SimpleDB SQL vs NoSQL (…) SQL 2008 Data storage capacity SQL vs NoSQL (…) GridFS stores files in two collections: chunks stores the binary chunks For details, see The chunks Collection files stores the file’s metadata For details, see The files Collection SQL vs NoSQL (…) The files Collection The chunks Collection BSON Types SQL vs NoSQL (…) Big Data Security • • • • • • Secure computations in distributed programming frameworks Security best practices for non-relational data stores Secure data storage and transactions logs Cryptographically enforced access control and secure communication Granular access control Real-time security/compliance monitoring Big Data Security (…) Technical Recommendations for sercurity • • • • • • • • Use Kerberos for node authentication Use file layer encryption Data anonymization Use key management Deployment validation Use secure communication Tokenization Cloud database controls Big data trends • Big data – of the people, by the • • • • • people, for the people Big data and social computing Cloud computing In memmory computing Mobile Applications and HTML5 Internet and big data Demo with MongoDB & Ref docs Ref docs: Judith Hurwitz, Alan Nugent, Dr Fern Halper, and Marcia Kaufman: Big Data For Dummies John Wiley & Sons, Inc 2013 “Technology Trends for 2013” prepared by Kaushal Amin, Chief Technology Officer, KMS Technology – Atlanta, GA, USA Website: http://hadoop.apache.org/ Demo with MongoDB Thank You !