TensorFlow World 2019 | Day 1 Livestream


TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TEST TENSORFLOW WORLD.>>BEN LORICA: GOOD MORNING! WELCOME TO DAY 1 OF KEY NOTES HERE. THE O’REILLY TEAMS ARE EXCITED TO HOLD OUR OWN FIRST EVER TENSORFLOW HERE IN SANTA CLAR AND WE WELCOME OUR LIVE STREAM AUDIENCE JOINING US FROM OVER THE WORLD.>>EDD WILDER-JAMES: WELCOME. IT IS GREAT TO HAVE YOU HERE AS PART OF THE TENSORFLOW COMMUNITY. BEFORE WE START, I WANTED TO TAKE A MOMENT TO RECOGNIZE ALL OF THOSE THAT WERE EFFECTED BY THE WILDFIRES IN NORTHERN CALIFORNIA AT THE MOMENT MANY HAVE BEEN IMPACTED, INCLUDING CONFERENCE TEAM AND SPEAKERS. THANK YOU FOR WORKING AT THIS DIFFICULT TIME. THE SPEEDY WAY IN WHICH THE TEAM HAS BOUNCED BACK FROM CHALLENGING CIRCUMSTANCES, OUR THOUGHTS ARE WITH YOU. WE HAVE ALREADY HAD A FEW TERRIFIC DAYS OF TRAININGS AND TUTORIALS HERE AT TENSORFLOW WORLD ALONG WITH A COULD CONTRIBUTOR CENTER AND A LIVELY FUN NIGHT AND POSTER EVENTS LAST NIGHT. WE HOPED YOU LEARNED SOMETHING NEW AND HAD A GREAT TIME. TO THE EARLY RISERS THAT JOINED US AT 6:30 THIS MORNING FOR THE AK FUN RUN AND SPEED NETWORKING EVENTS. THANK YOU!>>WE THANK OUR SPONSORS FOR THEIR GENEROUS SUPPORT OF TENSORFLOW WORLD. THEIR PARTICIPATION, IT IS MUCH APPRECIATED AND HELPS TO MAKE THIS CONFERENCE A TRULY GREAT EVENT.>>BEN LORICA: IF YOU’RE TWEETING, BLOGING, THE OFFICIAL #IS TFWORLD. SEND US A TWEET SO WE KNOW WHERE YOU’RE WATCHING FROM. WE INVITE YOU TO JOIN THAT STREAM AND TELL THE STORY OF THE CONFERENCE TO THOSE THAT CAN’T BE HERE.>>EDD WILDER-JAMES: WE’RE LIVE STREAMING AND CAPTIONING OUR KEY NOTES TODAY. WE’LL ALSO BE STREAMING ALL OF THE SESSIONS FROM THE CORE TECHNOLOGY TRACK TODAY AND TOMORROW. PLEASE SHARE THE LIVE STREAM LINK. OUR OPENING SKI NOTE SPEAKER THIS MORNING IS JEFF DEAN, HE’S A SENIOR FELLOW IN THE GOOGLE RESEARCH GROUP WHERE HE COULD YOU FOUNDED AND LEADS THE GOOGLE BRAIN TEAM. THEY’RE WORKING ON SYSTEMS FOR SPEECH RECOGNITION, COMPUTER VISION, LANGUAG UNDERSTANDING AND VARIOUS OTHER MACHINE LEARNING TASKS. JEFF CODESIGNED AND IMPLEMENTED MANY GENERATIONS OF GOOGLE’S CRAWLING INDEXING AND QUERY SYSTEM SERVICES, MAJOR PIECES OF THE ADVERTISING CONTENT SYSTEMS AND DISTRIBUTED INFRASTRUCTURE, INCLUDING BIG TABLE AND SPANNER SYSTEMS, PROTOCOL BUFFERS AND OTHER SYSTEM INFRASTRUCTURES FOR STATISTICAL MACHINE TRANSLATION. MOST IMPORTANTLY FOR US THIS WEEK, JEFF LED THE DEVELOPMENT AND OPEN SOURCING OF TENSORFLOW WITH THE INTENT THAT THE WHOLE WORLD HAVE ACCESS TO MACHINE LEARNING AT SCALE. PLEASE WELCOME JEFF DEAN.>>JEFF DEAN: THANK YOU! I’M EXCITED TO BE HERE! I THINK IT WAS FOUR YEARS AGO TO THE DAY THAT WE WERE ABOUT 20 PEOPLE SITTING IN A SMALL CONFERENCE ROOM AND ONE OF THE GOOGLE BUILDINGS, WE HAD WON UP EARLY TO TIME IT FOR AN EARLY EAST COAST LINE FOR EVERYONE, WE WERE TURNING ON THE TENSORFLOW WEBSITE, RELEASING THE FIRST VERSION EVER TENSORFLOW AS A OPEN SOURCE PROJECT. I’M REALLY, REALLY EXCITED TO SEE WHAT IT HAS BECOME. IT IS JUST REMARKABLE TO SEE THE GROWTH AND ALL OF THE DIFFERENT KINDS OF WAYS IN WHICH PEOPLE HAVE USED THIS SYSTEM FOR ALL KINDS OF INTERESTING THINGS AROUND THE WORLD. ONE THING THAT’S INTERESTING, IT IS THE GROWTH IN THE USE OF TENSORFLOW AND IT MIRRORS THE GROWTH AND INTEREST IN MACHINE LEARNING AND MACHINE LEARNING RESEARCH GENERALLY AROUND THE WORLD. THIS IS A GRAPH SEWING THE NUMBER OF MACHINE LEARNING ARCHIVE PAPERS THAT HAVE BEEN POSTED OVER THE LAST TEN YEARS OR SO. YOU SEE IT IS GROWING QUITE RAPIDLY, MUCH MORE QUICKLY THAN YOU MAY EXPECT. THAT LOWER REDLINE IS KIND OF THE NICE DOUBLING EVERY COUPLE OF YEARS GROWTH RATE WE WERE USED TO IN COMPUTING POWER DUE TO MOORE’S LAW FOR SO MANY YEARS. THAT HAS SLOWED DOWN. YOU SEE THAT THE MACHINE LEARNING RESEARCH COMMUNITY IS GENERATING RESEARCH IDEAS AT FASTER THAN THAT RATE. IT IS REMARKABLE. WE HAVE REPLACED COMPUTATIONAL GROWTH WITH GROWTH OF IDEAS. WE’LL SEE THOSE TOGETHER, THEY’LL BOTH BE IMPORTANT. THE EXCITEMENT ABOUT MACHINE LEARNING, IT IS BECAUSE WE CAN NOW DO THINGS WE COULDN’T DO BEFORE. AS LITTLE AS 5, 6 YEARS AGO, COMPUTERS REALLY COULDN’T SEE THAT WELL. STARTING IN ABOUT 2012, 2013 WE STARTED TO HAVE PEOPLE USE DEEP NEURAL NETWORKS TO TRY TO TACKLE COMPUTER VISION PROBLEMS, IMAGE CLASSIFICATION, DETECTION, STUFF LIKE THAT. NOW USING DEEP LEARNING AND DEEP NEURAL NETWORKS, YOU FEED IN THE RAW PIXELS OF AN IMMACK AND RELIABLY GET A PREDICTION OF WHAT’S IN THAT IMAGE. FEED THE PIXELS THERE, RED, GREEN, BLUE VALUES, COORDINATES, AND YOU GET OUT THE PREDICTION LEOPARD. THIS WORKS FOR SPEECH AS WELL, YOU FEED IN AUDIO WAVE FORMS, BY TRAINING A LOT OF THE WAVE FORMS AND TRANSCRIPTS IT, WE CAN TAKE A COMPLETELY NEW RECORDING AND TELL YOU WHAT IS BEING SAID. YOU CAN EVEN COMBINE THE IDEAS AND HAVE MODELS THAT TAKE IN PIXELS AND INSTEAD OF PREDICTING CLASSIFICATION, CLASSIFICATIONS OF WHAT IS IN THE OBJECT, YOU CAN WRITE A SHORT SENTENCE, A SHORT CAPTION THAT A HUMAN MAY WRITE ABOUT THE IMAGE, A CHEETAH LAYING ON TOP OF THE CAR. THAT’S A VACATION PHOTO. IT WAS KIND OF COOL. SO JUST TO SHOW THE PROGRESS IN COMPUTER VISION, IN 2011 STANFORD HOSTS AN IMAGE CONTEST EVERY YEAR TO SEE HOW WELL COMPUTER VISION SYSTEMS CAN PREDICT ONE OF 1,000 CATEGORIES IN A FULL COLOR IMAGE. YOU GET A MILLION TO TRAIN ON, THEN YOU GET TEST IMAGES THAT THE MODEL NEVER SAW BEFORE AND YOU HAVE TO MAKE A PREDICTION. IN 2011, THE WINNING BRANT GOT 26% ERROR. YOU CAN KIND OF MAKE OUT WHAT THAT IS. IT IS HARD TO TELL. WE KNOW FROM HUMAN EXPERIMENT THAT HUMAN ERROR OF A WELL-TRAINED HUMAN, SOMEONE THAT PRACTICED THIS UNDERSTANDS THE THOUSANDS OF CATEGORIES, THEY GET 5% ERROR, NOT A TRIVIAL TASK. IN 2016, THE WINNING BRANT GOT 3% ERROR. LOOK AT THAT TREMENDOUS PROGRESS IN THE ABILITY OF COMPUTERS TO RESOLVE AND UNDERSTAND COMPUTER IMAGERY AND HAVE COMPUTER VISION THAT ACTUALLY WORKS. THIS IS REMARKABLY IMPORTANT IN THE WORLD. NOW WE HAVE SYSTEMS THAT CAN PERCEIVE THE WORLD AROUND US AND WE CAN DO ALL KINDS OF REALLY INTERESTING THINGS ABOUT. WE HAVE SEEN SIMILAR PROGRESS IN SPEECH RECOGNITION, LANGUAGE TRANSLATION, THINGS LIKE THAT. FOR THE REST OF THE TALK, I WOULD LIKE TO STRUCTURE IT AROUND THIS NICE LIST OF 14 CHALLENGES THAT THE U.S. NATIONAL ACADEMY OF ENGINEERING PUT OUT, FELT LIKE THESE WERE IMPORTANT THINGS FOR SCIENCE AND TECHNOLOGY COMMUNITIES TO WORK ON FOR THE NEXT 100 YEARS, THEY PUT IT OUT IN 2008 AND CAME UP WITH A LIST OF 14 THINGS. AFTER DELIBERATION, AND I THINK YOU’LL AGREE THAT THESE ARE PRETTY GOOD, LARGE CHALLENGING PROBLEMS IF WE ACTUALLY MADE PROGRESS ON THEM, WE WOULD ACTUALLY HAVE A LOT OF PROGRESS IN THE WORLD. WE’LL BE HEALTHIER, ABLE TO LEARN THINGS BETTER, ABLE TO DEVELOP BETTER MEDICINES, HAVE ALL KINDS OF INTERESTING ENERGY SOLUTIONS. I’LL TALK ABOUT A FEW OF THESE. THE FIRST ONE I’LL TALK ABOUT, RESTORING AND IMPROVING URBAN INFRASTRUCTURE. WE’RE ON THE CUSP OF THE WIDE-SPREAD COMMERCIALIZATION OF REALLY INTERESTING NEW TECHNOLOGY THAT WILL CHANGE% HOW WE THINK ABOUT TRANSPORTATION IS IS IS IT LOOKS LIKE THIS IS NOW COMMERCIALLY VIABLE. WE HAVE COMPUTER VISION, LEARNING TECHNIQUES THAT CAN TAKE IN RAW FORMS OF DATA THAT THE CENTERS ON THESE CARS COLLECT THEY HAVE CAMERAS, RADAR IN THE FRONT BUMPER, REAR BUMPER, THEY CAN REALLY TAKE ALL OF THIS RAW INFORMATION IN AND WITH A DEEP NEURAL NETWORK FUSE IT TOGETHER TO BUILD A HIGH-LEVEL UNDERSTANDING OF WHAT’S GOING ON AROUND THE CAR. IS IT A CAR ON THE SIDE, A PEDESTRIAN UP TO THE LEFT, A LIGHT POST OVER HAD THERE I DON’T NEED TO WORRY ABOUT THAT MOVING, AND REALLY HELP TO UNDERSTAND THE ENVIRONMENT IN WHICH THEY’RE OPERATING AND THEN WHAT ACTIONS CAN THEY TAKE IN THE WORLD THAT ARE BOTH LEGAL, SAFE, OBEY ALL OF THE TRAFFIC LAWS AND GET THEM FROM A TO B. THIS IS NOT SOME DISTANT FAR OFF DREAM. TESTS HAVE BEEN RAN IN PHOENIX, ARIZONA, NORMALLY WITH A TEST THEY HAVE A SAFETY DRIVER IN THE FRONT SEAT READY TO TAKEOVER IF THE CAR DOES SOMETHING UNEXPECTED, FOR THE LAST YEAR OR SO THEY HAVE BEEN RUNNING TESTS IN PHOENIX WITH REAL PASSENGERS IN THE BACKSEAT AND NO SAFETY DRIVERS IN THE FRONT SEAT RUNNING AROUND SUBURBAN PHOENIX. SO SUBURBAN PHOENIX IS A SLIGHTLY EASIER TRAINING GROUND THAN MANHATTAN, SAN FRANCISCO, IT IS STILL SOMETHING THAT IS NOT REALLY FAR OFF. IT IS SOMETHING THAT’S ACTUALLY HAPPENING. THIS IS REALLY POSSIBLE BECAUSE OF THINGS LIKE MACHINE LEARNING AND THE USE OF TENSORFLOW IN THE SYSTEMS. ANOTHER ONE I’M REALLY, REALLY EXCITED ABOUT, ADVANCE HEALTH, THIS IS A BROAD AREA. THERE IS LOTS AND LOTS OF WAYS THAT MACHINE LEARNING DATA CAN BE USED TO MAKEE- BETTER HEALTHCARE DECISIONS FOR PEOPLE. I’LL TALK ABOUT ONE OF THEM. REALLY, I THINK THE POTENTIAL HERE, IT IS THAT WE CAN USE MACHINE LEARNING TO BRING THE WISDOM OF EXPERTS THROUGH A MACHINE LEARNING MODEL ANYWHERE IN THE WORLD. THAT’S REALLY A HUGE, HUGE OPPORTUNITY. SO LET’S LOOK AT THIS THROUGH ONE PROBLEM WE HAVE BEEN WORKING ON FOR A WHILE, DIABETIC RETINOPAPFY, T SCREENING EVERY YEAR, IF YOU’RE AT RISK, IF YOU HAVE DIABETES, EARLY SORT OF SYMPTOMS THAT MAKE IT LIKELY YOU MAY DEVELOP DIABETES, YOU SHOULD GET SCREENED EVERY YEAR. THERE ARE WILL 400 MILLION PEOPLE AROUND THE WORLD THAT SHOULD BE SCREENED EVERY YEAR. THE SCREENING, IT IS SPECIALIZED, DOCTORS CAN’T DO IT, YOU NEED AN OPHTHALMOLOGY LEVEL OF TRAINING TO DO THIS EFFECTIVELY. THE IMPACT OF THE SHORTAGE IS SIGNATURE. SIGNIFICANT. IN INDIA, THERE IS A SHORTAGE AND AS A RESULT 45% OF PATIENTS WHO ARE DIAGNOSED WITH THIS DISEASE HAVE SUFFERED EITHER FULL OR PARTIAL VISION LOSS BEFORE THEY’RE DIAGNOSED AND THEN TREATED. THIS IS TRAGIC BECAUSE IF YOU CATCH IT IN TIME, IT IS TREATABLE, THERE IS A 99% FORM OF TREATMENT THAT WE HAVE TO MAKE SURE THAT THE RIGHT PEOPLE ARE TREATED AT THE RIGHT TIME. SO WHAT CAN YOU DO? IT TURNS OUT THE SCREENING, IT IS A COMPUTER VISION PROBLEM, AND THE PROGRESS WE HAVE MADE ON GENERAL COMPUTER VISION PROBLEMS WHERE YOU WANT TO TAKE A PICTURE, TELL IF THAT’S A LEOPARD, AN AIRCRAFT CARRIER, CAR, IT WORKS WITH THIS. SO YOU TAKE THIS IMAGE, THE SCREENING CAMERA, IT SORT OF IS THE RAW DATA AND YOU TRY TO FEED THAT IN A MODEL PREDICTING 1, 2, 3, 4, 5, THAT’S HOW THEY’RE GRADED. ONE, NO ISSUE, 5 5 HIGH AND E OTHER NUMBER IN BETWEEN. YOU CAN GET A COLLECTION OF DATA AND HAVE OPHTHALMOLOGISTS LABEL THEM. TURNS OUT IF YOU ASK TWO TO LABEL THE SAME IMAGE, THEY AGREE 60% OF THE TIME ON THE NUMBER 1, 2, 3, 4, 5, PERHAPS SLIGHTLY SCARIER, IF YOU ASK THE SAME PERSON TO GRADE THE SAME IMAGE A FEW HOURS AGO, THEY AGREE WITH THEMSELVES 65% OF THE TIME. YOU CAN FIX IT BY GETTING EACH IMAGE LABELED, SO YOU GET IT LABELED, IT IS MORE OF A 2, 3, IT EVENTUALLY YOU HAVE A ME DATASET YOU CAN TRAIN ON, HIGH QUALITY DATA, GREED IE RIGHT RAW GREED CAN’T WANT AND YOU APPLY AN OFF THE SHELF COMPUTER MODEL BASED ON THIS DATASET. NOW YOU GET A MODEL THAT’S ON PAR OR PERHAPS SLIGHTLY BETTER THAN THE AVERAGE BOARD CERTIFIED OPHTHALMOLOGIST IN THE U.S. THAT’S AMAZING. TURNS OUT YOU CAN DO BETTER THAN THAT. IF YOU GET THE DATA LABELED BY SPECIALISTS, THOSE WITH MORE TRAINING IN THIS DISEASE AND CHANGE THE PROTOCOL BY WHICH YOU LABEL THINGS, YOU GET 3 SPECIALISTS TO LOOK AT AN IMAGE, DISCUSS IT AMONGST THEMSELVES AND COME UP WITH WHAT’S CALLED A COORDINATED ASSESSMENT AND ONE NUMBER, AND THEN YOU CAN TRAIN A MODEL, NOW BE ON PAR WITH SPECIALISTS, THE GOLD STANDARD OF CARE IN THIS AREA. THAT’S SOMETHING THAT YOU CAN NOW TAKE AND DISTRIBUTE WIDELY AROUND THE WORLD. ONE ISSUE PARTICULARLY WITH HEALTHCARE KIND OF PROBLEMS, IT IS YOU WANT EXPLAINABLE MODELS, YOU WANT TO EXPLAIN TO A CLINICIAN WHY IS THIS PERSON — WHY DO WE THINK THIS PERSON HAS MODERATE RETINOPATH. Y, YOU TAKE THIS E IMAGE, YOU SHOW WHY THIS IS A 2, NOT A 3, BY HIGHLIGHTING THIS, YOU CAN MAKE IT MORE UNDERSTANDABLE FOR CLINICIANS AND ENABLE THEM TO REALLY SORT OF GET BEHIND THE ASSESSMENT THAT THE MODEL IS MAKING. WE HAVE SEEN THIS IN OTHER AREAS AS WELL. THERE’S A LOT OF WORK ON EXPLAINABILITY. THE NOTION THAT DEEP NEURAL NETWORKS ARE COMPLETE BLACK BOXES IS A BIT OVERDONE. THERE ARE A BUNCH OF GOOD TECHNIQUES BEING DEVELOPED AND MORE ALL THE TIME THAT WILL IMPROVE THIS. A BUNCH OF ADVANCES DEPEND ON BEING ABLE TO UNDERSTAND TEXT. WE HAVE HAD A LOT OF REALLY GOOD IMPROVEMENTS IN THE LAST FEW YEARS ON LANGUAGE UNDERSTANDING. THIS IS A BIT OF A STORY OF RESEARCH AND HOW IT BUILDS ON OTHER RESEARCH. IN 2019, RESEARCHERS, INTERNS, THEY CAME UP WITH A NEW MODEL FOR TEXTS CALLED THE TRANSFORMER MODEL. UNLIKE RECURRENT MODELS WHERE YOU HAVE A SEQUENTIAL PROCESS WHERE YOU ABSORB ONE WORD, ONE TOKEN AT A TIME AND UPDATE SOME INTERNAL STATE AND THEN GO ON TO THE NEXT TOKEN, THIS MODEL ENABLES YOU TO PROCESS A WHOLE BUNCH OF TEXT ALL AT ONCE IN PARALLEL MAKING IT MUCH MORE COMPUTATIONALLY EFFICIENT AND TO USE A TENSION ON PREVIOUS TEXTS TO REALLY FOCUS ON IF I’M TRYING TO PREDICT WHAT THE NEXT WORD IS, YOU KNOW, WHAT ARE OTHER PARTS OF THE CONTEXT TO THE LEFT THAT ARE RELEVANT TO PREDICTING THAT? THAT PAPER WAS QUITE SUCCESSFUL AND SHOWED REALLY GOOD RESULTS ON LANGUAGE TRANSLATION TASKS WITH A LOT LESS COMPUTE. THE BLUE SCORE THERE, THE FIRST TWO COLUMNS FROM ENGLISH TO GERMAN, FRENCH, HIGHER IS BETTER, AND THEN THE COMPUTE COST OF THE MODELS, IT SHOWS THAT THIS IS GETTING STATE-OF-THE-ART RESULTS AT THAT TIME WITH 10 TO 100 MORE COMPUTES THAN OTHER APPROACHES. IN 2018, ANOTHER TEAM OF GOOGLE RESEARCHERS BUILT ON THE IDEA OF TRANSFORMERS SO EVERYTHING YOU SEE IN THE BLUE OVAL IS A TRANSFORMER MO DO YOU LIVE AND THEY COME UP WITH THIS APPROACH, BUY DIRECTIONAL ENCODER, ENCODER REPRESENTATIONS FROM TRANSFORMERS. BERT IS SHORTER AND MORE CATCHY! SO BERT HAS A NICE PROPERTY THAT IN ADDITION TO USING CONTEXT TO THE LEFT, IT USES CONTEXT ALL AROUND THE LANGUAGE, THE SURROUNDING TEXT IN ORDER TO MAKE PREDICTIONS ABOUT TEXT. THE WAY IT WORKS, YOU START WITH A SELF-SUPERVISED OBJECTIVE. ONE REALLY NICE THING ABOUT THIS, THERE IS LOTS AND LOTS OF TEXT IN THE WORLD. IF YOU CAN FIGURE OUT A WAY TO USE THAT TEXT TO TRAIN A MODEL TO BE ABLE TO UNDERSTAND TEXT BETTER, THAT WOULD BE GREAT. WE’LL TAKE THIS TEXT AND IN THE BURT TRAINING OBJECTIVE, WE’LL DROP ABOUT 15% OF THE WORDS. THIS IS ACTUALLY PRETTY HARD. THE MODEL IS THEN GOING TO TRY TO FILL IN THE BLANKS ESSENTIALLY. TRY TO PREDICT WHAT ARE THE MISSING WORDS THAT ARE DROPPED. BECAUSE WE HAVE THE ORIGINAL WORDS, WE NOW KNOW IF THE MODEL IS CORRECT AND IT GUESSES WHAT’S IN THE BOX. BY PROCESSING TRILLIONS OF WORDS OF TEXT LIKE THIS, YOU GET A VERY GOOD UNDERSTANDING OF CONTECTUAL QUEUES AND LANGUAGE AND HOW TO ACTUALLY FILL IN THE BLANKS IN A REAL INTELLIGENT WAY. SO THAT’S ESSENTIALLY THE TRAINING OBJECTIVE TORE BERT, YOU TAKE THE TEXT, DROP 15% OF IT, YOU TRY TO PREDICT THE MISSING WORDS. ONE KEY THING THAT WORKS REALLY WELL, IT IS THAT’S STEP 1, YOU PRETRAIN A MODEL ON LOTS AND LOTS OF TEXT, USING THIS FILL IN THE BLANK SELF-SUPERVISED OBJECTIVE FUNCTION. STEP 2, YOU THEN TAKE A LANGUAGE TASK YOU REALLY CARE ABOUT, MAYBE YOU WANT TO PREDICT IS THIS A FIVE-STAR REVIEW, A ONE-STAR REVIEW FOR SOME HOTEL AND YOU HAVE VERY MUCH LABELED TEXT FOR THAT LABELED TASK. YOU HAVE THE STAR COUNT OF EACH REVIEW. YOU CAN THEN FINE TUNE THE MODEL STARTING WITH MODEL TRAIN IN STEP 1 ON TRILLIONS OF WORDS OF TEXT AND THEN USE THE PALTRY 10,000 EXAMPLES FOR THE TEXT TASK YOU REALLY CARE ABOUT AND THAT WORKS EXTREMELIAL WELL. IN PARTICULAR, BERT GAVE STATE-OF-THE-ART RESULTS ACROSS A BROAD RANGE OF DIFFERENT TEXT UNDERSTANDING BENCHMARKS IN THIS GLUED BENCHMARK SUITE WHICH IS PRETTY COOL. PEOPLE HAVE BEEN USING BERT IN THIS WAY TO IMPROVE ALL KINDS OF DIFFERENT THINGS ALL ACROSS THE LANGUAGE UNDERSTANDING IN THIS SPACE. ONE OF THE CHALLENGES WAS 9ENGINEERING THE TOOLS OF SCIENTIFIC DISCOVERY. MACHINE LEARNING IS ACTUALLY GOING TO BE AN IMPORTANT COMPONENT OF MAKING ADVANCES IN GRAND CHALLENGE AREAS, THINGS LIKE YOU A TON MISS VEHICLES, OTHER KINDS OF THINGS, AND IT HAS BEEN REALLY SATISFYING TO SEE WHAT WE HOPED WOULD HAPPEN WHEN WE RELEASED THIS PROJECT, AND IT ACTUALLY CAME TO PASS AS WE WERE HOPING IN THAT LOTS OF PEOPLE WOULD KIND OF PICK UP TENSORFLOW, USE IT FOR ALL KINDS OF THINGS, PEOPLE WOULD IMPROVE THE CORESYSTEM, USE IT FOR TASKS WE NEVER WOULD IMAGINE AND THAT’S BEEN QUITE SATISFYING. PEOPLE HAVE DONE ALL KINDS OF THINGS, SOME OF THESE ARE USES INSIDE OF GOOGLE, SOME ARE OUTSIDE, AND ACADEMIC SITUATIONS, SOME ARE SCIENTISTS WORKING ON CONSERVING WHALES OR UNDERSTANDING ANCIENT SCRIPTS, MANY KINDS OF THINGS, IT IS PRETTY NEAT. THE BREADTH OF USE, IT IS REALLY AMAZING. THESE ARE THE 20 WINNERS OF THE GOOGLE.ORG AI IMPACT CHALLENGE WHERE PEOPLE COULD SUBMIT POWERPLAYS OF HOW THEY USE MACHINE LEARNING AND AI TO TACKLE A LOCAL CHALLENGE THEY SAW IN THE COMMUNITIES AND THEY HAVE ALL KINDS OF THINGS RANGING FROM TRYING TO PREDICT BETTER AMBULANCE DISPATCHING TO IDENTIFYING ILLEGALLOGGING USING SPEECH RECOGNITION OR AUDIO PROCESSING, PRETTY NEAT. MANY OF THEM ARE USING TENSORFLOW. ONE OF THE THINGS WE’RE EXCITED ABOUT, IT IS AUTO EVAL. IT IS AN IDEA OF AUTOMATING SOME OF THE PROCESS BY WHICH MACHINE LEARNING EXPERTS SIT DOWN AND SORT OF MAKE DECISIONS TO SOLVE MACHINE LEARNING PROBLEMS. CURRENTLY YOU HAVE A MACHINE LEARNING EXPERT SIT DOWN, THEY TAKE DATA, THEY HAVE COMPUTATION, THEY RUN EXPERIMENTS, THEY STIR THEM ALTOGETHER, EVENTUALLY YOU 9 HAVE A SOLUTION TO A PROBLEM THAT YOU ACTUALLY CARE ABOUT. ONE OF THE THINGS WE WOULD LIKE TO DO THOUGH, SEE IF WE CAN ELIMINATE A LOT OF THE NEED FOR THE HUMAN MACHINE EXPERT TO RUN THE EXPERIMENTS AND AUTOMATE THE EXPERIMENTAL PROCESS BY WHICH AN EXPERT COMES BY A HIGH-QUALITY SOLUTION FOR A PROBLEM WE CARE ABOUT. LOTS AND LOTS OF ORGANIZATIONS AROUND THE% WORLD HAVE MACHINE LEARNING PROBLEMS. MANY, MANY OF THEM DON’T EVEN REALIZE THEY HAVE A MACHINE LEARNING PROBLEM, LET ALONE HAVE PEOPLE IN THE ORGANIZATION THAT CAN TACKLE THE PROBLEM. ONE OF THE EARLIEST PIECES OF WORK OUR RESEARCHERS DID IN THIS SPACE IS CALLED NEURAL ARCHITECTURE SEARCH. WHEN YOU SIT DOWN, DESIGN THIS NETWORK TO TACKLE A PARTICULAR TASK, YOU MAKE A LOT OF DECISIONS ABOUT SHAPES OF THIS, THAT, 3 BY 3 FILTERS, LAYER 17, 5 BY 5, ALL KINDS OF THINGS LIKE THIS. IT TURNS OUT THAT YOU CAN AUTOMATE THIS PROCESS BY HAVING A MODEL GENERATING MODEL AND TRAINING THAT MODEL GENERATING MODEL BASED ON FEEDBACK ON HOWAL WITH THE MODELS THAT IT GENERATES WORKS ON THE PROBLEM YOU CARE ABOUT. THE WAY IT WORKS, WE GENERATE THE MODELS, THOSE ARE DESCRIPTIONS, AND YOU TRAIN EACH OF THOSE FOR A FEW HOURS. THEN WE’LL SEE HOW WELL THEY WORK. THEN YOU USE THE ACCURACY OF THE MODELS AS A RE-INFORCEMENT LEARNING SIGNAL FOR THE MODEL GENERATING MODEL TO STEER IT AWAY FROM MODELS THAT DIDN’T WORK WELL AND TOWARDS MODELS THAT WORKED BETTER. WE’LL REPEAT MANY, MANY TIMES. OVER TIME, WE’LL GET BETTER AND BETTER BY STEERING THE SEARCH TO THE PARTS OF THE SPACE OF MODELS THAT WORKED WELL. IT COMES UP WITH MODELS THAT LOOK A LITTLE STRANGE ADMITTEDLY, A HUMAN WILL NOT SIT DOWN AND WEAR UP A MACHINE LEARNING COMPUTER VISION MODEL EXACTLY THAT WAY. THEY’RE PRETTY EFFECTIVE. IF YOU LOOK AT THIS GRAPH, THIS SHOWS KIND OF THE BEST HUMAN MACHINE LEARNING EXPERTS, COMPUTER VISION EXPERT, MACHINE LEARNING RESEARCHERS IN THE WORLD PRODUCING A BUNCH OF DIFFERENT KINDS OF MODELS IN THE LAST FOUR, FIVE YEARS, ALL KINDS OF THINGS, THAT BLACK DOTTED LINE, IT IS KIND OF THE FRONTIER OF HUMAN MACHINE LEARNING EXPERT MODEL QUALITY ON THE Y AXIS AND COMPUTATIONAL COST ON THE X AXIS. WHAT YOU SEE, YOU GO OUT, THE X AXIS, YOU GET MORE ACCURACY BUT USE APPLYING MORE COMPUTATIONAL COST. WHAT YOU SEE, IT IS THE BLUE DOTTED LINE, IT IS AUTO MLB BASED SOLUTIONS, SYSTEMS WHERE WE DID THIS AUTOMATED EXPERIMENTS INSTEAD OF PREDESIGNING ARCHITECTURE, IT IS BETTER AT THE HIGH END, YOU CARE ABOUT THE MOST ACCURATE MODEL YOU CAN GET REGARDLESS OF COSTS AND IT IS ACCURATE AT THE LOW END WHERE YOU CARE ABOUT A LIGHT WEIGHT MODEL THAT MAY RUN ON A PHONE OR SOMETHING LIKE THAT. IN 2019 WE HAVE BEEN ABLE TO IMPROVE THAT SIGNIFICANTLY. 9 THIS IS A SET OF MODELS CALLED EFFICIENT AND IT HAS A SLIDER ABOUT IT, YOU CAN TRADE-OFF THE COMPUTATIONAL COST AND ACCURACY. THEY’RE ALL WAY BETTER THAN HUMAN SORT OF GUIDED EXPERIMENTATION ON THE BLACK DOTTED LINE THERE. THIS IS TRUE FOR IMAGE RECOGNITION, CLASSIFICATION, IT IS GOOD FOR OBJECT DETECTION, THE REDLINE THERE, IT IS AUTO ML, THE OTHER THINGS ARE NOT. IT IS TRUE FOR LANGUAGE TRANSLATION, SO THE BLACK LINE THERE, IT IS VARIOUS KINDS OF TRANSFORMERS, THE REDLINE, IT IS WE GAVE THE BASIC COMPONENTS OF TRANSFORMERS TO AN AUTO ML SYSTEM AND ALLOWS THAT TO FIDDLE WITH IT, COMING UP WITH SOMETHING BETTER. IT IS TRUE FOR COMPUTER VISION MODELS, USED IN YOU A TON MISS VEHICLES, THIS IS A COLLABORATION THAT WE CAME UP WITH MODELS THAT ARE SIGNIFICANTLY LOWER LATENCY FOR THE SAME QUALITY OR WE TRADED OFF AND GOT SIGNIFICANTLY LOWER ERROR RATE AT THE SAME LATENCY. IT WORKS FOR TAB LAYOLA DATA, YOU HAVE CUSTOMER RECORDS AND YOU WANT TO PREDICT WHAT CUSTOMERS SPEND 1,000 WITH THE BUSINESS NEXT MONTH, YOU CAN USE AUTO ML TO COME UP WITH A HIGH-QUALITY MODEL FOR THAT KIND OF PROBLEM. WHAT DO WE WANT? WE WANT THE FOLLOWING PROPERTY IN A MACHINE LEARNING MODEL. ONE, WE TEND TO TRAIN SEPARATE MODELS FOR EACH DIFFERENT MODEL WE CARE ABOUT. THIS IS A BIT MISGUIDED, WE WANT ONE MODEL DOING A LOT OF THINGS SO IT BUILDS ON THE KNOWLEDGE IN HOW IT DOES THOUSANDS OR MILLIONS OF DELIVER THINGS SO THAT WHEN THE MILLION AND FIRST THING COMES ALONG, YOU CAN ACTUALLY USE THE EXPERTISE WITH ALL OF THE OTHER THINGS THEY KNOW HOW TO GET INTO THAT WITH RELATIVELY LITTLE DATA. THESE ARE NICE PROPERTIES, I HAVE A CARTOON DIAGRAM OF SOMETHING THAT MAY MAKE SENSE. IMAGINE WE HAVE A MODEL LIKE THIS. IT IS SPARSELY ACTIVATED, DIFFERENT PIECES OF THE MODEL HAVE DIFFERENT TYPES OF EXPERTISE AND THEY’RE CALLED UPON WHEN IT MAKES SENSE AND THEY’RE MOSTLY IDLE. IT IS POWER DEFICIENT BUT CAN DO MANY THINGS. NOW EACH COMPONENTS HERE, SOME PIECE OF A MACHINE LEARNING MODEL WITH DIFFERENT KIND OF STATE, PARAMETERS INVOLVED, DIFFERENT OPERATIONS, AND A NEW TASK COMES ALONG. NOW, YOU CAN IMAGINE SOMETHING LIKE NEURAL ARCHITECTURE SEARCH BECOMING SQUINTED AT IT JUST RIGHT AND TURNING INTO A NEURAL PATHWAY SEARCH, YOU CAN LOOK FOR COMPONENTS THAT ARE REALLY GOOD FOR THIS NEW TASK WE CARE ABOUT AND MAYBE WE’LL SEARCH AND FIND THAT THIS PATH THROUGH THE MODEL GETS US INTO A GOOD STATE FOR THE NEW TASK BECAUSE MAYBE IT GOES THROUGH COMPONENTS THAT ARE TRAINED ON RELATED TASKS ALREADY. NOW MAYBE WE WANT THAT MODEL TO BE MORE ACCURATE FOR THE PURPLE TASK. SO WE ADD A BIT MORE COMPUTATIONAL CAPACITY, ADD A NEW COMPONENT, START TO USE THAT COMPONENT FOR THE NEW TASK, CONTINUE TRAINING IT, AND NOW THAT NEW COMPONENT CAN ALSO BE USED FOR SOLVING OTHER RELATED TASKS. EACH COMPONENT ITSELF MAY BE RUNNING SOME SORT OF INTERESTING ARCHITECTURAL SEARCH INSIDE IT. SOMETHING LIKE THAT, IT IS THE DIRECTION WHICH WE’LL BE EXPLORING AS A COMMUNITY. IT IS NOT WHAT WE’RE DOING TODAY. IT COULD BE PRETTY INTERESTING, THE DIRECTION. FINALLY, I WOULD LIKE TO TOUCH ON THOUGHTFUL USE OF AI DESIGN. AS WE HAVE SEEN MORE AND MORE USES OF MACHINE LEARNING, IN OUR PRODUCTS, AROUND THE WORLD, IT IS REALLY, REALLY IMPORTANT TO BE THINKING CAREFULLY ABOUT HOW WE WANT TO APPLY THESE TECHNOLOGIES. LIKE ANY TECHNOLOGY, THE SYSTEMS CAN BE USED FOR AMAZING THINGS OR THINGS WE MAY FIND A LITTLE SORT OF DETRIMENTAL IN VARIOUS WAYS. WE HAVE COME UP WITH A SETS OF PRINCIPLES BY WHICH WE THINK ABOUT APPLYING SORT OF MACHINE LEARNING AND AI TO OUR PRODUCTS AND WE HAVE MADE THESE PUBLIC ABOUT A YEAR AND A HALF AGO AS A WAY OF SHARING OUR THOUGHT PROCESS WITH THE REST OF THE WORLD. I PARTICULARLY LIKE THESE. I POINTED OUT MANY ARE AREAS OF RESEARCH THAT ARE NOT FULLY UNDERSTOOD YET, WE AIM TO APPLY THE BEST IN THE STATE-OF-THE-ART METHODS, FOR EXAMPLE, FOR REDUCING BIAS IN MACHINE LEARNING MODELS AND ALSO CONTINUING TO BE DO RESEARCH AND ADVANCE THE STATE-OF-THE-ART IN THESE AREAS. THIS IS KIND OF A TASTE OF DIFFERENT KINDS OF WORK WE’RE DOING IN THIS AREA, HOW DO WE DO MACHINE LEARNING WITH MORE PRIVACY USING THINGS LIKE FEDERATED LEARNING, HOW DO WE MAKE MODELS MORE INTERPRET RUMBLE SO THAT THE CLINICIAN CAN UNDERSTAND THE PREDICTIONS ITS MAKING ON DIABETIC RETINOPATHY EXAMPLES, HOW DO WE MAKE MACHINE LEARNING MORE FAIR. WITH THAT, I HOPE I HAVE CONVINCED YOU THAT DEEP NEURAL NETS AND MACHINE LEARNING, YOU’RE ALREADY HERE, YOU’RE ALREADY CONVINCED OF THIS, THEY’RE HELPING TO MAKE SIGNIFICANT ADVANCES IN A LOT OF HARD COMPUTER PROBLEMS, LANGUAGE UNDERSTANDING, GENERAL USE OF MACHINE LEARNING, IT WILL PUSH THE WORLD FORWARD. THANK YOU VERY MUCH. I APPRECIATE YOU ALL BEING HERE.>>BEN LORICA: THANK YOU, JEFF, FOR A GREAT TOUR OF THE APPLICATIONS THAT ARE BEING BUILT AND SKETCHING OUT WHAT FUTURE NEURAL ARCHITECTURES MAY LOOK LIKE. AN IMPORTANT COMPONENT OF ANY CONFERENCE IS NETWORKING. FOR THE NEXT FEW DAYS, THIS IS YOUR TENSORFLOW COMMUNITY, WE HAVE MANY GREAT SPEAKER, NOT JUST FROM GOOGLE, GOOGLE TENSORFLOW, BUT FROM MANY DIFFERENT COMPANIES. ALSO YOUR FELLOW ATTENDEES MAY BE SOURCES OF INSPIRATION AS WELL. SOME OF THE THINGS THAT JEFF DESCRIBED MAY BE BUILT BY PEOPLE WHO ARE IN ATTENDANCE AT THIS CONFERENCE. IF YOU MISSED OUT ON SPEED NETWORKING BEFORE KEY NOTES TODAY, THERE IS ANOTHER CHANCE TOMORROW MORNING FROM 8:00 TO 8:30 A.M. SPEED NETWORKING WILL TAKE PLACE RIGHT OUTSIDE HERE AT KEY NOTES IN THE MISSION CITY BALLROOM FOYER. PLEASE CHECK OUT THE CODE OF CONDUCT, IT IS LINKED ON THE CONFERENCE HOME PAGE. IT IS AN IMPORTANT PART OF THE CONFERENCE EXPERIENCE. WE INVITE EVERYONE TO HELP MAKE THIS A WELCOMING, RESPECTFUL ENVIRONMENT FOR EVERYONE HERE THIS WEEK. OUR NEXT SPEAKER,S FREDERICK REISS, HEAD OF PRODUCT FOR Twitter CENTRAL MACHINE LEARNING ORGANIZATION. HE’S A PRODUCT MANAGER WORKING ACROSS APPLIED RESEARCH ML SERVICES AND MACHINE LEARNING. PLEASE WELCOME THEODORE SUMMER OF Twitter. INCHES I’M THEODORE SUMMER FROM Twitter. BEFORE I GET STARTED WITH MY CONVERSATION TODAY, IT I WANT TO DO A QUICK PLUG FOR Twitter. WHAT’S GREAT ABOUT EVENTS LIKE THIS, YOU HEAR PEOPLE LIKE JEFF DEAN TALK. YOU ALSO GET TO HEAR FROM COLLEAGUES AND PEOPLE IN THE INDUSTRY THAT ARE FACING SIMILAR CHALLENGES AS YOU AND HAVE CONVERSATIONS AROUND DEVELOPMENTS AND DATA SCIENCE AND MACHINE LEARNING. WHAT’S GREAT, IT IS AVAILABLE EVERY DAY ON Twitter, Twitter IS IF HE NOMINAL ON CONVERSATION ON DATA SCIENCE AND MACHINE LEARNING, YOU CAN FOLLOW THE CONVERSATIONS AND ENGAGE IN THEM BUT YOU CAN BRING THAT CONVERSATION BACK TO YOUR WORKPLACE AND COME OFF LOOKING LIKE A HERO. JUST SOMETHING TO CONSIDER. WITHOUT THAT SHAMELESS PLUG, I’M THEODORE SUMMER, I LEAD PRODUCT FOR CORTEX, THE Twitter MACHINE LEARNING ORGANIZATION. IF YOU HAVE QUESTIONS FOR ME, THE TEAM, CATCH UP WITH ME ON Twitter AND WE’LL FOLLOW-UP LATER. BEFORE WE GET INTO HOW WE’RE ACCELERATING ML AT Twitter, LET’S TALK ABOUT HOW WE’RE EVEN USING ML AT Twitter. Twitter IS LARGELY ORGANIZED AGAINST THREE CUSTOMER NEEDS, THE FIRST OF WHICH OUR HEALTH INITIATIVE. THAT MAY BE A LITTLE BIT CONFUSING TO YOU, YOU MAY THINK OF IT AS USER SAFETY, WE THINK ABOUT IT AS IMPROVING THE HEALTH OF CONVERSATIONS ON Twitter. MACHINE LEARNING IS ALREADY AT USE HERE. WE USE IT TO DETECT SPAM. WE CAN ALGORITHM PROTECT SPAM AND PROTECT USERS. SIMILARLY, IN THE ABUSE SPACE, WE CAN PROACTIVELY FLAG CONTENT AS POTENTIALLY ABUSE, TOSS IT UP FOR HUMAN REVIEWS AND ACT ON IT BEFORE THE USERS ARE EVEN IMPACTED BY IT. A THIRD SPACE WHERE WE’RE USING MACHINE LEARNING HERE IS NSFW, NOT SAFE FOR WORK. YOU’RE ALL FAMILIAR WITH THAT ACRONYM. HOW CAN WE AT SCALE IDENTIFY THIS CONTENT AND HANDLE IT ACCORDINGLY. ANOTHER USE OF MACHINE LEARNING IN THIS SPACE. HERE THE CUSTOMER JOB, WHAT WE’RE SERVING, CONNECTING OUR CUSTOMERS WITH A CONVERSATION ON Twitter THAT THEY’RE INTERESTED IN. A PRIMARY VEIN WE DO THAT, IT IS A TIMELINE. IT IS RINGED, USERS FOLLOW ACCOUNTS, CONTENTS AND TWEET THE ACCOUNTS, THEY’RE FUNNELED INTO A CENTRAL FEED AND WE RANK THAT BASED ON YOUR PAST EXPERIENCE TO BRING FORTH THE MOST DEVELOPED AND RELEVANT CONVERSATIONS FOR YOU. NOW, THERE IS LOTS OF CONVERSATIONS ON Twitter, YOU’RE NOT FOLLOWING EVERYTHING. THERE IS A JOB THAT WE HAVE TO SERVE ABOUT BRINGING FORTH ALL THE CONVERSATIONS YOU’RE NOT PROACTIVELY FOLLOWING BUT ARE STILL RELEVANT. THIS IS ON THE RECOMMENDATION PRODUCT WHICH IS A MACHINE LEARNING TO SCAN THE CONTENT ON Twitter AND IDENTIFY WHAT CONVERSATIONS WOULD BE MOST INTERESTING TO YOU AND PUSH IT TO YOU IN A NOTIFICATION.>>THE INVERSE OF THAT, YOU KNOW WHAT THE TOPICS YOU WANT TO EXPLORE ARE. YOU’RE LOOKING FOR CONVERSATIONS AROUND THAT. THIS IS WHERE YOU USE Twitter SEARCH. THIS IS ANOTHER SURFACE AREA IN THE BIG BLUE APP THAT WE’RE USING MACHINE LEARNING. THE THIRD JOB FOR THE CUSTOMERS, CONNECTING BRANDS WITH THE CUSTOMERS. YOU MAY THINK OF IT AS THE ADS PRODUCT, THIS IS THE OG OF MACHINE LEARNING AT Twitter. THE FIRST TEAM THAT I.ED IT. HERE YOU USE IT FOR AD RANKING, THE TIMELINE RANKING BUT INSTEAD OF TWEETS IT IS ADS AND IDENTIFYING THE RELEVANT ADS FOR USERS. AS SIGNALS THAT GO INTO THAT, YOU HAVE USER TARGETING TO UNDERSTAND THE PAST ENGAGEMENT, UNDERSTANDING WHAT ADS ARE EARNING YOUR INTEREST SPACE. THE THIRD IS BRAND SAFETY, YOU MAY NOT THIS ABOUT THIS WITH MACHINE LEARNING AND ADVERTISING, IF YOU’RE A COMPANY LIKE UNITED, YOU WANT TO ADVERTISE ON Twitter, YOU WANT TO MAKE SURE THAT THE AD NEVER SHOWS UP NEXT TO A TWEET ABOUT A PLANE CRASH HOW DO WE PROTECT OUR BRANDS FOR THIS? WE USE MACHINE LEARNING FOR THIS AS WELL. AS YOU CAN TELL, MACHINE LEARNING IS A BIG PART OF ALL OF THESE ORGANIZATIONS TODAY AND WHERE WE HAVE SHARED INTEREST AND SHARED INVESTMENT, WE WANT TO MAKE SURE WE HAVE A SHARED ORGANIZATION THAT SERVES THEM. THAT’S THE NEED FOR CORTEX. OF CORTEX IS THE CENTRAL MACHINE LEARNING TEAM AND THE CONTEXT IS SIMPLE, TO ENTER Twitter WITH ADVANCED AI AND WE HAVE ORGANIZED IN THREE WAYS. THE FIST, APPLIED RESEARCH GROUP, THIS GROUP APPLIES THE MOST ADVANCED ML TECHNIQUES FROM INDUSTRY AND RESEARCH TO OUR MOST IMPORTANT SURFACE AREAS WHETHER NEW INITIATIVES OR EXISTING PLACES. THIS TEAM IS AN INTERNAL TASKFORCE OR CONSULT TANSY TO REDEPLOY AGAINST THE TOP INITIATIVES. SIGNALS, SHARED ASSETS THAT ARE BROADLY USEFUL PROVIDES LEVERAGE. EXAMPLES OF THIS IS LANGUAGE UNDERSTANDING TEAM THATIC LOOS AT TWEETS AND IDENTIFIES NAMED ENTITIES INSIDE OF THEM. THEY CAN THEN BE OFFERED UP AS FEATURES FOR OTHER TEAMS TO CONSUME IN THEIR OWN APPLICATIONS OF MACHINE LEARNING. SIMILARLY, OUR MEDIA UNDERSTANDING TEAM LOOKS AT IMAGES AND CREATES A FINGERPRINT OF ANY IMAGE AND THEREFORE WE CAN IDENTIFY EVERY USE OF THAT IMAGE ACROSS THE PLATFORM. THESE ARE EXAMPLES OF SHARED SIGNALS WE’RE PRODUCING USED FOR MACHINE LEARNING AT SCALE INSIDE OF THE COMPANY. THE THIRD, PLATFORM TEAM. THIS IS REALLY THE ORIGINS OF CORTEX. HERE WE PROVIDE TOOLS AND INFRASTRUCTURE TO ACCELERATE ML DEVELOPMENT AT Twitter. WE’RE INCREASING THE INVEST OF OUR M, WILL PRACTITIONERS. THIS IS REALLY THE FOCUS OF THE CONVERSATION TODAY. WHEN WE SET OUT TO BUILD THIS ML PLATFORM, WE DECIDED WE WANTED TO SHARE THE ML PLATFORM ACROSS ALL OF Twitter. WHY IS THAT IMPORTANT THAT IT IS SHARED OVER Twitter? WE WANT TRANSFERABILITY, WE THE ADS TEAM TO BE TRANSFERABLE TO BENEFIT THE HEALTH INITIATIVE WHERE THAT IS RELEVANT. SIMILARLY, IF WE HAVE GREAT TALENT IN THE CONSUMER TEAM, THAT’S INTERESTED IN MOVING TO THE ADS TEAM, IF THEY’RE ON THE SAME PLATFORM, THEY TRANSFER WITHOUT FRICTION AND RAMP UP QUICKLY. SO HE WITH SET OUT WITH THIS GOAL OF HAVING A SHARED ML PLATFORM ACROSS ALL OF Twitter. WHEN WE DID THAT, WE LOOKED AT A COUPLE OF PRODUCT REQUIREMENTS. FIRST, NEEDS TO BE SCALABLE, NEEDS TO BE ABLE TO OPERATE AT Twitter SCALE. THE SECOND, IT NEEDS TO BE ADAPTABLE. THIS SPACE, IT IS DEVELOPING QUICKLY. WE NEED A PLATFORM THAT CAN EVOLVE WITH DATA SCIENCE AND MACHINE LEARNING DEVELOPMENTS. THIRD, IT IS THE TALENT POLE. WE WANT TO MAKE SURE WE HAVE A DEVELOPMENT ENVIRONMENT AT Twitter THAT APPEALS TO THE RESEARCHERS AND ENGINEERS THAT WE’RE HIRING AND DEVELOPING. FOURTH, IT IS THE ECOSYSTEM. WE WANT TO BE ABLE TO LEAN ON THE PARTNERS THAT ARE DEVELOPING INDUSTRY LEADING TOOLS SO THAT WE CAN FOCUS ON TECHNOLOGIES THAT ARE Twitter SPECIFIC. FOURTH, DOCUMENTATION. YOU UNDERSTAND THAT. WE WANT TO UNBLOCK OUR PRACTITIONERS AS THEY HIT ISSUES, WHICH IS INEVITABLE ON NIP PLATFORM. FINALLY, USABILITY. WE WANT TO REMOVE FRICTION AND FRUSTRATION FROM THE LIVES OF OUR TEAM. THEY CAN FOCUS ON DELIVERING VALUE FOR OUR END CUSTOMERS. CONSIDERING THESE PRODUCT REQUIREMENTS, LET’S SEE HOW TENSORFLOW HAS DONE AGAINST THEM. SCALABILITY, WE HAVE USED TENSORFLOW WITH DEEP BIRD AGAINST TIMELINE RANKING SO EVERY TWEET THAT’S RANKED IN THE TIMELINE TODAY RUNS THROUGH TENSORFLOW. WE CAN CONSIDER THAT TEST VALIDATED. SECOND, ADAPTABILITY. THE NOVEL ARCHITECTURE THAT CAN BE SUPPORTED AND THE CUSTOM LOSS FUNCTIONS ALLOWS US TO REACT TO THE LATEST RESEARCH AND OUTSIDE OF THE COMPANY. THE EXAMPLE WE PUBLISHED ON THIS PUBLICLY IS THE USE OF A SPLIT NOTE ARCHITECTURE IN AD RANKING, TENSORFLOW IS ADAPTABLE FOR US. THIRD, TALENT POLL, YOU THINK OF THAT IN TWO TYPES. THERE IS THE ML ENGINEER AND RESEARCHER. AS A PROXY OF THE AUDIENCES, WE LOOKED AT THE GET HUB DATA ON THIS AND THIS IS WIDELY ADOPTED AMONG ENGINEERS. THE ARCHIVE COMMUNITY SHOWS STRONG EVIDENCE OF OPTIONS IN THE ACADEMIC COMMUNITY. ON TOP OF THIS PROXY DATA, WE HAVE ALSO HAD EVIDENCE OF THE SPEED OF RAMP UP FOR OUR RESEARCHERS AND ML ENGINEERS INSIDE OF THE COMPANY. FOURTH, IT IS THE ECOSYSTEM, WHETHER IT IS TENSER BOARD, DATA VALIDATION, MODEL ANALYSIS, TF META STORE, TFX PIPELINES, THERE IS THESE PRODUCTS OUT THERE THAT ARE IF HE NOMINAL, THEY ALLOW US TO FOCUS ON DEVELOPING TOOLS AND INFRASTRUCTURE THAT’S SPECIFIC TO Twitter NEEDS AND LEAN ON THE GREAT WORK OF OTHERS. WE’RE GRATEFUL FOR THIS AND TENSORFLOW DOES GREAT HERE. THE FIFTH, DOCUMENTATION, NOW, THIS IS WHAT YOU WOULD GO TO WHEN YOU GO TO TENSORFLOW AND SEE THAT IF HE NOMINAL DOCUMENTATION AS WELL AS GREAT EDUCATION RESOURCES. WHAT YOU MAY NOT APPRECIATE AND WHAT WE HAVE COME TO REALLY APPRECIATE IS THE VALUE OF THE USER GENERATED CONTENT, WHAT IS STACK OVERFLOW AND OTHERS PROVIDE IN GENERATED CONTENT IS ALMOST AS VALUABLE AS WHAT TENSORFLOW CAN CREATE. GIVEN THE WIDE VET ADOPTION AND THE GREAT WEBSITE, THEY PROVIDED IF HE NOMINAL DOCUMENTATION FOR ML PRACTITIONERS. FINALLY, USABILITY. THIS IS WHAT WE’RE EXCITED ABOUT TENSORFLOW 2.0, THE ORGANIZATION AROUND THE API MAKES IT U.S.A.ER FRIENDLY AND CONTINUES TO ALLOW FOR FLEXIBILITY FOR ADVANCED USERS. THE EXECUTION ENABLES MORE RAPID AND INTUITIVE DEBUGGING AND CLOSES THE GAP BETWEEN ML EDGE ANOTHERS AND MODELERS. CLEARLY FROM THIS CHECKLIST, WE’RE HAPPY WITH OUR ENGAGEMENT WITH TENSORFLOW. WE’RE EXCITED ABOUT CONTINUING TO DEVELOP THE PLATFORM WITH THEM AND PUSH LIMITS ON WHAT IT CAN DO. WITH GRATITUDE TO THE COMMUNITY FOR THE PARTICIPATION IN THE PROJECT. IF YOU HAVE QUESTIONS FOR ME, CONNECT WITH ME, BUT I’M NOT ALONE HERE TODAY, A BUNCH OF MY COLLEAGUES ARE HERE AS WELL, IF YOU SEE THEM ROAMING THE HALLS, ENGAGE WITH THEM, OR AS I SHARED BEFORE, YOU CAN CONTINUE THE CONVERSATION ON Twitter. HERE ARE THEIR HANDLES, THANK YOU FOR YOUR TIME. CHEERS. I’M HAPPY TO SAY FRIENDS IN GHANA, LEBANON, ALL KINDS OF PLACES THAT WISHED US WELL THIS MORNING. KEEP THAT COMING. IT IS GREAT TO KNOW THAT TENSORFLOW WORLD IS TRULY GOING AROUND THE WORLD. AFTER THE KEYNOTE, PLEASE JOAN US FOR THIS MORNING’S BREAK. THAT WILL BE SERVED IN THE EXPO HALL NEXT TO REGISTRATION. PLEASE SHARE YOUR FEEDBACK BY RATING TODAY’S SESSIONS. YOU CAN RATE THAT ON ANY SESSION DETAIL PAGE ON THE WEBSITE AND LEAVE FEEDBACK OR QUESTIONS IN THE COMMENTS. WE HAVE A LOT OF GREAT ACTIVITIES HAPPENING IN AND AROUND THE CONFERENCE SPACE OVER THE NEXT TWO DAYS. PLEASE COME TO THE EXPO HALL, SEE THE LATEST FROM TENSORFLOW AND DEMO BOOTHS AND COME AND TALK WITH SPONSORS THAT ARE CONTRIBUTING TO THE WHOLE GROWING TENSORFLOW ECOSYSTEM. NOW OUR NEXT SPEAKER THIS MORNING IS FREDERICK REISS. HE’S FROM SAN FRANCISCO AND PERSONALLY A LONGSTANDING CONTRIBUTOR TO TENSORFLOW ITSELF. JOIN MEAN IN WELCOMING FREDERICK REISS.>>FREDERICK REISS: I WORK FOR IBM AND HAVE BEEN SINCE 2006 AND HAVE CONTRIBUTED TO TENSORFLOW SINCE 2017, MY PRIMARY JOB IS TO SERVE AS TECH LEAD FOR OPEN SOURCE DATA AND AI TECHNOLOGIES. WE’RE AN OPEN SOURCE LAB LOCATED IN DOWNTOWN SAN FRANCISCO AND WE WORK ON OPEN SOURCE TECHNOLOGIES THAT ARE FOUNDATIONAL TO AI. WE HAVE 44 FULL TIME DEVELOPERS WHO WORK ONLY ON OPEN SOURCE SOFTWARE. THAT’S A LOT OF OPEN SOURCE DEVELOPERS, OR IS IT? IF YOU LOOK AT THE IBM,S THAT ARE COMMITTED TO OPEN SOURCE, YOU WILL FIND THAT THERE ARE ALMOST 1200IBMERS IN THAT CATEGORY. OUR 44 DEVELOPERS ARE ACTUALLY A VERY SMALL SLICE OF A VERY LARGE PIE. THOSE NUMBERS, THEY DON’T INCLUDE RED HAT. WHEN WE CLOSE THAT ACQUISITION EARLIER THIS YEAR, WE MORE THAN DOUBLED OUR NUMBER OF ACTIVE CONTRIBUTORS TO OPEN SOURCE. SO YOU CAN SEE THAT IBM IS REALLY BIG IN OPEN SOURCE. MORE AND MORE, THE BULK OF OUR CONTRIBUTIONS IN THE OPEN ARE GOING TOWARDS THE FOUNDATIONS OF AI. WHEN I SAY AI, I MEAN AI IN PRODUCTION, I MEAN AI AT SCALE. AI AT SCALE IS NOT AN ALGORITHM, NOT A TOOL, IT IS A PROCESS. IT IS A PROCESS THAT STARTS WITH DATA, AND THEN THAT DATA TURNS INTO FEATURES, AND THE FEATURES, THEY TRAIN MODELS AND THE MODELS GET DEPLOYED IN APPLICATIONS AND THE APPLICATIONS PRODUCE MORE DATA AND THE WHOLE THING STARTS ALL OVER AGAIN. AT THE CORE OF THIS PROCESS, IT IS AN ECOSYSTEM OF OPEN SOURCE SOFTWARE. AT THE CORE OF THIS ECOSYSTEM, IT IS TENSORFLOW. THAT IS WHY I’M HERE ON BEHALF OF IBM OPEN SOURCE TO WELCOME YOU TO TENSORFLOW WORLD. THROUGHOUT THE CONFERENCE YOU WILL SEE TALKS THAT SPEAK TO ALL OF THE DIFFERENT STAGES OF THIS AE LIFE CYCLE. I THINK YOU’RE GOING TO SEE A SPECIAL EMPHASIS ON THIS PART, MOVING MODELS INTO PRODUCTION. ONE OF THE MOST IMPORTANT ASPECTS OF MOVING MODELS INTO PRODUCTION, IT IS THAT WHEN THE MODEL IS DEPLOYED IN A REAL WORLD APPLICATION, IT IS GOING TO START HAVING EFFECTS ON THE REAL WORLD. IT BECOMES IMPORTANT TO ENSURE THAT THOSE EFFECTS ARE POSITIVE AND THAT THEY’RE FAIR TO YOUR CLIENTS, TO YOUR USERS. NOW, AT IBM, HERE IS A HYPOTHETICAL EXAMPLE RESEARCHERS PUT TOGETHER A YEAR AGO TAKING REAL MEDICAL DATA AND PRODUCED A MODEL PREDICTING WHICH PATIENTS ARE MORE LIKELY TO GET SICK AND WILL GET ADDITIONAL SCREENING AND THEY SHOWED THAT IF YOU NAIVELY TRAIN THIS MODDEDAL YOU END UP WITH A MODEL WITH SIGNATURE BOY I CAN’T SAY, BY DEPLOYING STATE-OF-THE-ART TECHNIQUES TO ADJUST THE DATASET AND THE PROCESS OF MAKING THE MODEL THEY COULD SUBSTANTIALLY REDUCE THE MODEL TO PRODUCE A MODEL, REDUCE THE BIAS, PRODUCING A MODEL THAT’S MUCH MORE FAIR. YOU CAN SEE A JUNE TAR NOTEBOOK WITH AN SCENARIO FROM END-TO-END WITH CODE, EQUATIONS, RESULTS AT THE TIME URL DOWN HERE. AGAIN, I NEED TO EMPHASIZE, THIS WAS A HYPOTHETICAL EXAMPLE. WE BUILT A FLAWED MODEL DELIBERATELY SO WE CAN SHOW HOW TO MAKE IT BETTER. NO PATIENTS WERE HARMED IN THIS EXERCISE. HOWEVER, LAST FRIDAY I SAT DOWN WITH MY MORNING COFFEE, I OPENED UP THE WALLSTREET JOURNAL AND I SAW THIS ARTICLE AT THE BOTTOM OF PAGE 3. IT DESCRIBES A SCENARIO EARLY SIMILAR TO OUR HYPOTHETICAL. YOU KNOW, WHEN YOUR HYPOTHETICAL STARTS TO SHOW UP AS NEWSPAPER HEADLINES, THAT’S KIND OF SCARY. I THINK IT IS INCUMBENT UPON US AS AN INDUSTRY TO MOVE FORWARD THE PROCESS, THE TECHNOLOGY OF TRUSTED AI, TRUST AND TRANSPARENCY IN AI WHICH IS WHY IBM AND IBM RESEARCH RELEASED OUR TOOL KITS OF STATE-OF-THE-ART ALGORITHMS IN THE SPACE AS OPEN SOURCE UNDER AI FAIRNESS 360, AI EXPLAINABILITY 36 OF 0 AND ADVERSARIAL ROBUSTNESS 360. IT IS WHY IBM IS WORKING WITH OTHER MEMBERS OF THE LENOX FOUNDATION TRUSTED AI COMMITTEE TO MOVE FORWARD OPEN STANDARDS IN THIS AREA SO THAT WE CAN ALL MOVE QUICKLY, MORE QUICKLY TO TRUSTED AI. CAN TALK TO YOU ABOUT COOL OPEN SOURCE DEMOS AND CHECK OUT THE WEBSITE IT’S, DEVELOPER IBM AND KODAK.ORG. ON BEHALF OF IBM, I WELCOME YOU ALL TO TENSORFLOW WORLD. ENJOY THE CONFERENCE. THANK YOU.>>EDD WILDER-JAMES: I ENCOURAGE YOU TO VISIT THAT WEBSITE. THEY ARE REALLY PUSHING OUT A LOT OF REALLY USEFUL, FRACTURE CAL OPEN SOURCE PROJECTS IN DEVELOPER RESOURCES. PLEASE JOIN US FOR LUNCH TODAY IN THE EXPO HALL. SOME OF THE TABLES AT LUNCH WILL HAVE SIGNS WITH INDUSTRIES AND TOPICS FOR DISCUSSION. FEEL FREE TO ENJOY ONE OF THE LUNCHTIME TOPIC TABLES. OUR THANKS TO GOOGLE CLOUD FOR SPONSORING TODAY’S LUNCH. ALSO PLEASE CHECK OUT THE INCLUSION LOUNGE IN THE EX POPE HALL FOR CAREER COACHING FROM SENIOR GOOGLE ENGINEERS, ALIED SKILL ADVICE AND LEARNING MORE ABOUT SUPPORTING PARTNER ORGANIZATIONS THAT ARE INCREASING DIVERSITY IN THE TECH COMMUNITY. OUR NEXT SPEAKER, MEGAN KACHOLIA. SHE’S VP OF ENGINEERING WITHIN GOOGLE’S RESEARCH ORGANIZATION. HER TEAMWORK EXPANDS AT THE LEARNING IN RESEARCH AND INTRODUCTION, SUCH AS PRODUCTS LIKE TENSORFLOW. SHE HAD A LONG TEE NEWER IN THE GOOGLE AD ORGANIZATION WHERE SHE RAN THE SERVING SYSTEM FOR GOOGLE’S DISPLAY AD BUSINESS. PLEASE WELCOME MEGAN KACHOLIA.>>MEGAN KACHOLIA: HEY, EVERYONE. GOOD MORNING. JUST WANT TO SAY, FIRST OF ALL, WELCOME. TODAY I WANT TO TALK A LITTLE BIT ABOUT TENSORFLOW 2.0 AND SOME OF THE NEW UPDATES WE HAVE THAT WILL MAKE YOUR EXPERIENCE WITH TENSORFLOW EVEN BETTER. BEFORE I DIVE IN TO A LOT OF THE DETAILS, I WANT TO START OFF BY THANKING YOU. EVERYONE HERE, EVERYONE ON THE LIVE STREAM, EVERYONE THAT’S BEEN CONTRIBUTING TO TENSORFLOW, ALL OF YOU WHO MAKE UP THE COMMUNITY. TENSORFLOW WAS OPEN SOURCED TO HELP ACCELERATE THE AI FIELD FOR EVERYONE. YOU HAVE USED IT IN YOUR EXPERIMENTS, YOU DEPLOYED IT MADE AMAZING DIFFERENTU HAVE- APPLICATIONS THAT WE’RE SO EXCITED TO SHOWCASE AND TALK ABOUT, SOME THAT WE GET TO SEE A BIT HERE TODAY, IT IS ONE OF MY FAVORITE PARTS ABOUT CONFERENCES LIKE THIS. YOU HAVE DONE SO MUCH MORE. ALL OF THIS HAS HELPED MAKE TENSORFLOW WHAT IT IS TODAY. THE MOST POPULAR ECOSYSTEM IN THE WORLD. HONESTLY, THAT WOULD NOT HAPPEN WITHOUT THE COMMUNITY BEING EXCITED AND EMBRACING AND USING THIS AND GIVING BACK. ON BEHALF OF THE ENTIRE TEAM, I REALLY JUST FIRST WANT TO SAY THANK YOU. IT IS AMAZING TO SEE HOW TENSORFLOW IS USED, THAT’S ONE OF THE GREATEST THINGS I GET TO SEE ABOUT MY JOB, THE APPLICATIONS AND THE WAY FOLKS ARE USING TENSORFLOW. I WANT TO TAKE A STEP BACK AND TALK A LITTLE BIT ABOUT SOME DIFFERENT USER GROUPS AND HOW WE SEE THEM MAKING USE OF TENSORFLOW WE’RE CAMING OUT RESEARCHERS, DATA SCIENTISTS, DEVELOPERS AND THERE ARE OTHER GROUPS IN BETWEEN AS WELL. RESEARCHERS USE IT BECAUSE IT IS FLEXIBLE. IT IS FLEXIBLE ENOUGH TO EXPERIMENT WITH AND PUSH THE STATE-OF-THE-ART IN DEEP LEARNING. YOU HEARD THIS EVEN A FEW MINUTES AGO WITH FOLKS ON Twitter TALKING ABOUT HOW THEY’RE ABLE TO USE TENSORFLOW AND EXPAND ON TOP OF IT IN ORDER TO DO SOME OF THE AMAZING THINGS THAT THEY WANT TO MAKE USE OF ONLY THEIR OWN PLATFORM. AT GOOGLE WE SEE EXAMPLES OF THIS WHEN RESEARCHERS ARE CREATING ADVANCED MODELS AND THINGS THAT JEFF REFERENCED N A TALK EARLIER. TAKING A STEP FORWARD, LOOKING AT DATA SCIENTISTS, DATA SCIENTISTS AND ENGINEERS SAY THEY RELY ON TENSORFLOW FOR PERFORMANCE AND SCALE, THAT’S A BIG THING ABOUT TENSORFLOW WE HAVE EMPHASIZED AND LOOKED AT FROM THE BEGINNING, HOW CAN WE MAKE SURE THIS SCALES TO LARGE PRODUCTION USE CASES? FOR EXAMPLE, QUANTIFY AND BLACK ROCK USE TENSORFLOW TO TEST AND DEPLOY BERT IN REAL WORLD INSTANCES SUCH AS TEXT TOKENIZATION AND CLASSIFICATION. HELPING ONE STEP FORWARD, LOOKING AT APPLICATION DEVELOPERS, APPLICATION DEVELOPERS USE TENSORFLOW BECAUSE IT IS EASY TO LEARN ML ON PLATFORMS THAT THEY CARE ABOUT. WE WANT TO MAKE ML SIMPLE, MICRO CONTROLLERS, SO THAT THEY RELY ON THE PRETRAINED MODELS AND MICRO FOR DEPLOYMENT. EACH OF THE GROUPS, IT IS A CRITICAL PART OF THE TENSORFLOW ECO SYSTEM. THIS IS WHY WE REALLY WANTED TO MAKE SURE THAT TENSORFLOW 2.0 WORKS FOR EVERYONE. WE ANNOUNCED THE ALPHA IN OUR DEV SUMMIT EARLIER THIS YEAR AND OVER THE PAST FEW MONTHS THE TEAM HAS BEEN WORKING VERY HARD TO INCORPORATE EARLY FEEDBACK, THANK YOU TO THE COMMUNITY FOR GIVING US THE EARLY FEEDBACK SO WE CAN MAKE SURE WE’RE DEVELOPING SOMETHING THAT WORKS WELL FOR YOU, AND WE HAVE BEEN WORKING TO RESOLVE BUGS AND ISSUES AND THINGS LIKE THAT. JUST LAST MONTH, IN SEPTEMBER, WE WERE EXCITED TO ANNOUNCE THE FINAL GENERAL RELEASE FOR TENSORFLOW 2.0. YOU MAY BE FAMILIAR WITH TENSER FLOW’S ARCHITECTURE WHICH IS ALWAYS SPORTING THE ML LIFE CYCLE FROM TRAINING THROUGH DEPLOYMENT. A A THING WE HAVE EMPHASIZED SINCE THE BEGINNING SINCE IT WAS OFFICIALLY OPEN SOURCED A FEW YEARS AGO. I WANT TO EMPHASIZE HOW 2.0 MAKES THIS WORKFLOW EVEN EASIER AND MORE INTUITIVE. FIRST, WE INVESTED IN USING AN EASY TO USE PACKAGE IN TENSORFLOW, THE DEFAULT HIGH-LEVEL API. MANY DEVELOPERS LOVE THIS BECAUSE IT IS EASY TO USE AND TO UNDERSTAND. AGAIN, YOU HEARD THIS ALREADY MENTIONED A LITTLE BIT EARLIER AND HOPEFULLY WE’LL HEAR MORE ABOUT IT THROUGH THE NEXT FEW DAYS. BY TIGHTLY INTEGRATING THIS INTO 2.0 WE CAN MAKE IT WORK EVEN BETTER WITH PRIMITIVES LIKE TF DATA, WE CAN DO PERFORMANCE OPTIMIZATIONS BEHIND THE SCENES AND RUN DISTRIBUTED TRAINING. WE WANTED 2.0 TO FOCUS ON USABILITY, HOW DO WE MAKE IT EASIER FOR DEVELOPERS? MAKING IT EASIER FOR USERS TO GET WHAT THEY NEED OUT OF TENSORFLOW? FOR INSTANCE, A CUSTOMIZE WEIGHT LOSS APP SAY THEY USE THIS FOR DESIGNING THEIR NETWORK BY LEVERAGING STRATEGY DISTRIBUTION IN 2.0 THEY WERE ABLE TO UTILIZE THE FULL POWER OF THEIR GPUS. IT IS FEEDBACK LIKE THIS THAT WE LOVE TO HEAR. AGAIN, IT IS VERY IMPORTANT FOR US TO KNOW HOW THE COMMUNITY IS MAKING USE OF THINGS, HOW THE COMMUNITY IS USING 2.0, THINGS THEY WANT TO SEE SO THAT WE CAN MAKE SURE WE’RE DEVELOPING THE RIGHT FRAMEWORK AND MAKE SURE YOU CONTRIBUTE BACK. WHEN YOU NEED CONTROL TO CREATED A ADVANCED ALGORITHMS, WE HAVE EAGER ANTICIPATION MAKING IT FAMILIAR FOR PYTHON DEVELOPERS AND THIS IS ESPECIALLY USEFUL WHEN YOU’RE STEPPING THROUGH, DOING DEBUGGING, MAKING SURE THAT YOU REALLY UNDERSTAND STEP BY STEP WHAT’S HAPPENING, THIS ALSO MEANS THAT THERE IS LESS CODING REQUIRED WHEN TRAINING THE MODEL, ALL WITHOUT HAVING TO USE SESSION.RUN. AGAIN, USABILITY, IT IS A FOCUS. TO DEMONSTRATE THE POWER OF TRAINING MODELS WITH 2.0 I’LL SHOW YOU HOW TO TRAIN A STATE-OF-THE-ART MODEL IN TEN LINES OF CODE USING THE TRANSFORMERS LIBRARY BY A COMMUNITY CONTRIBUTION. THIS HAS SOME OF THE MOST ADVANCED OPTIONS AVAILABLE TODAY. LOOKING THROUGH THE CODE, YOU CAN USE THIS FOR TEXT CLASSIFICATION. YOU CAN SEE JUST SIMPLY LOAD THE TOKENIZER, MODEL, DATASET, THEN YOU PREPARE THE DATASET, YOU USE TF AND COMPILE AND FIT APIS AND WITH A FEW LINES OF CODE I CAN TRAIN THE MODEL. YOU NEED PERFORMANCE TO MATCH FOR THIS. THAT’S WHY WE WORKED HARD TO IMPROVE PERFORM FANS WITH 2 2.0. IT DELIVERS UP TO 3X FASTER TRAINING PERFORMANCE WITH MIXED PRECISION IN A FEW LINES OF CODE WITH MODELS LIKE THIS. AS WE CONTINUE TO DOUBLE DOWN ON 2.0 IN THE FUTURE, PERFORMANCE WILL REMAIN A FOCUS WITH MORE MODELS AND WITH HARDER ACCELERATORS. FOR EXAMPLE, IN 2.1, THE UPCOMING TENSORFLOW RELEASE, YOU EXPECT TPU AND CPU SUPPORT ALONG WITH MIXED PRECISION FOR GPUS. PERFORMANCE IS SOMETHING THAT WE KEEP A FOCUS ON AS WELL WHILE MAKING SURE THAT USERABILITY STANDS TO THE FOREFRONT. THERE IS A LOT MORE TO THE ECOSYSTEM, SO BEYOND MODEL BUILDING AND PERFORMANCE, THERE ARE MANY OTHER PIECES THAT HELP ROUND OUT THE TENSORFLOW ECOSYSTEM. ADD ONS AND EXTENSIONS, THEY’RE VERY IMPORTANT PIECES HERE, WHICH IS WHY WE WANTED TO MAKE SURE THAT THERE ARE — THAT THEY’RE COMPATIBLE WITH TENSORFLOW 2 2.0, YOU HAVE USE POPULAR LIBRARIES LIKE THOSE CALLED OUT HERE, WHETHER IT IS PROBABILITY, TF AGENTS, TEXTS, WE HAVE INTRODUCED A HOST OF NEW LIBRARIES TO HELP RESEARCHERS AND ML PRACTITIONERS IN MORE USEFUL WAYS. FOR EXAMPLE, NEURAL STRUCTURE LEARNING HELPS TO TRAIN NEURAL NETWORKS WITH STRUCTURED SIGNALS, AND THEN THE FAIRNESS INDICATORS ADD ON ENABLES REGULAR COMPUTATION AND VISUALIZATION OF THE FAIRNESS MATRIX. THIS IS TYPES OF THE THINGS THAT YOU CAN SEE KIND OF AS PART OF THE TENSORFLOW ECOSYSTEM AND THEY HELP YOU MAKE SURE THAT YOU’RE ABLE TO DO THE THINGS YOU NEED TO DO, NOT WITH MODELS BUT BEYOND JUST THAT. ANOTHER VALUABLE ASPECT OF THE TENSORFLOW ECOSYSTEM IS ABLE TO ANALYZE YOUR ML EXPERIMENTS IN DETAIL. THIS IS SHOWING TENSER BOARD, IT IS THE VISUALIZATION TOOLKIT WHICH HELPS YOU TO ACCOMPLISH THIS. THIS IS A POPULAR TOOL AMONG RESEARCHERS AND ML PRACTITIONERS FOR TRACKING METRICS, VISUALIZING GRAPHS AND PARAMETERS AND MUCH MORE. IT IS INTERESTING THAT WE HAVE SEEN USERS ENJOY TENSER BOARD SO MUCH THAT THEY TAKE SCREEN SHOTS OF THE EXPERIMENTS AND USE THEM TO SHARE WITH OTHERS WHAT THEY’RE DOING WITH TENSORFLOW. THIS TYPE OF SHARING AND COLLABORATION IN THE ML COMMUNITY IS SOMETHING THAT WE REALLY WANT TO ENCOURAGE WITH TENSORFLOW. AGAIN, THERE IS SO MUCH THAT CAN HAPPEN BY ENABLING THE COMMUNITY TO DO GOOD THINGS. THAT’S WHY I’M EXCITED TO SHARE THE PREVIEW OF TENSER BOARD.DEV, A NEW, FREE, MANAGED TENSER BOARD EXPERIENCE THAT LET’S YOU UP BOARD AND SHARE THE RESULTS WITH ANYONE. YOU WILL NOW BE ABLE TO HOST AND TRACK YOUR ML EXPERIMENTS AND SHARE THEM PUBLICLY. NO SETUP REQUIRED. SIMPLY UP LOAD THE LOGS. THEN YOU SHARE THE URL SO THAT OTHER HADS CAN SEE THE EXPERIMENTS AND SEE THE THINGS THAT YOU ARE DOING WITH TENSORFLOW. AS A PREVIEW, WE’RE STARTING OFF WITH A SCALERS DASHBOARD, BUT OVER TIME WE’LL ADD A LOT MORE FUNCTIONALITY TO MAKE THE SHARING EXPERIENCE EVEN BETTER. IF YOU’RE NOT LOOKING TO BUILD MODELS FROM SCRATCH, YOU WANT TO REDUCE COMPUTATIONAL COSTS, TENSORFLOW HAS ALREADY MADE PRETRAINED TRAINING MODELS AVAILABLE THROUGH TENSORFLOW HUB AND WE’RE EXCITED TO SHARE AN IMPROVED EXPERIENCE OF TENSORFLOW HUB THAT’S MORE INTUITIVE WHERE YOU CAN FIND A COMPREHENSIVE REPOSITORY OF PRETRAINED MODELS IN THE ECOSYSTEM. THIS MEANS THAT YOU CAN FIND MODELS LIKE BERT AND OTHERS RELATED TO IMAGE, TEXT, VIDEO, MORE THAT ARE READY TO USE WITH TENSORFLOW LIGHT AND TENSORFLOW.JS. AGAIN, WE WANT TO MAKE SURE THAT THE EXPERIENCE HERE IS VASTLY IMPROVED TO MAKE IT EASIER FOR YOU TO FIND WHAT YOU NEED IN ORDER TO MORE QUICKLY GET TO THE TASK AT HAND. SENSE TENSORFLOW IS DRIVEN BY ALL OF YOU, TENSORFLOW HUB IS HOSTING MORE PRETRAINED MODELS FROM THE COMMUNITY. YOU’LL BE ABLE TO FIND CURATED MODELS ABOUT I DEEP MIND, GOOGLE, MICRO SOFT’S AI FOR EARTH AND OTHERS READY TO USE TODAY WITH MANY MORE TO COME. WE WANT TO MAKE SURE THAT TENSORFLOW HUB IS A GREAT PLACE TO FIND THE EXCELLENCE PRETRAINED MODELS AND AGAIN THERE IS SO MUCH THAT THE COMMUNITY IS DOING, WE WANT TO SHOWCASE THOSE MODELS AS WELL. 2.0 ALSO HIGHLIGHTS THE CORE STRENGTHS AND AREAS OF FOCUS WHICH IS BEING ABLE TO GO FROM MODEL BUILDING, EXPERIMENTATION THROUGH TO PRODUCTION NO MATTER THE PLATFORM YOU WORK ON. YOU CAN DEPLOY END-TO-END ML PIPELINES, YOU CAN USE MODELS ON MOBILE AND EMBEDDED DEVICES WITH TENSORFLOW LIGHT AND YOU CAN TRAIN AND RUN MODELS IN THE BROWSER OR OTHERS WITH THIS. YOU WILL LEARN MORE ABOUT WHAT’S NEW IN TENSORFLOW IN PRODUCTION DURING THE KEYNOTE SESSIONS TOMORROW. YOU CAN LEARN MORE ABOUT THESE UPDATES BY GOING TO TENSORFLOW.ORG WHERE YOU’LL FIND THE LATEST DOCUMENTATION, EXAMPLES, TUTORIALS FOR 2.0. AGAIN, WE WANT TO MAKE SURE IT IS EASY FOR OF THE COMMUNITY TO SEE WHAT’S HAPPENING, WHAT’S NEW, AND ENABLE YOU TO DO WHAT YOU NEED TO DO WITH TENSORFLOW. WE HAVE BEEN THRILLED TO THE POSITIVE RESPONSES TO 2.0 AND HOPE YOU CONTINUE TO SHARE YOUR FEEDBACK. THANK YOU. I HOPE YOU ENJOY THE REST OF TF WORLD.>>BEN LORICA: THANK YOU. THAT WAS AMAZING. I REALLY WANT TO ECHO, YOU KNOW, MY DAY-TO-DAY, IT IS WORKING WITH TENSORFLOW CONTRIBUTOR COMMUNITY AND WHAT MAKING — IT REALLY MAKES IT WORTH COMING TO WORK EVERY DAY, IT IS THE PEOPLE, THE COMMUNITY, IT IS ALWAYS THE CONNECTION AND THE WAY WE WORK TOGETHER THAT MAKES THIS PROJECT SO EXCITING TO ME. THANK YOU IF YOU’RE A PART OF THE COMMUNITY AND A CONTRIBUTOR TO TENSORFLOW. EVERYONE IS WELCOME TO JOIN THE BETTER TOGETHER DIVERSITY LUNCH TODAY THAT’S IN THE MAN MANGOLIA ROOM, WE’LL BE LEARNING ABOUT IDEAS ON SUPPORTING DIVERSITY IN TECH. NOW, THERE IS SO MUCH GREAT CONTENT HERE AT THE CONFERENCE. SESSIONS GOING ON AT THE SAME TIME, NO ONE COULD POSSIBLY CAPTURE IT ALL IN PERSON AT THE SAME TIME. THE GOOD NEWS, ALL OF THE KEY NOTES IT, SESSIONS HERE AT TENSORFLOW WORLD, THEY’LL BE AVAILABLE AFTER THE CONFERENCE, WE’LL POST THEM ON THE TENSORFLOW YOUTUBE CHANNEL AND ALSO AVAILABLE ON O’REILLY LEARNING. NOW, OUR NEXT SPEAKER THIS MORNING, IT IS CRAIG WILEY. CRAIG WILEY IS THE DIRECTOR OF PRODUCT FOR GOOGLE CLOUD’S AI PLATFORM, A CODE BASED ENVIRONMENT FOR ML DEVELOP TEMPERATURES AND DATA SCIENTISTS TAKING PROJECTS FROM IDEAS TO DEPLOYMENT, HE SPENT THE YEARS AT AMAZON AS GENERAL MANAGER OF THE SAGE MAKER. HELP MEAN IN WELCOMING CRAIG WILEY. — HELP ME. I WANT TO SAY I HAVE BEEN DABBLING IN CLOUD AI AND MACHINE LEARNING FOR A WHILE. DURING THAT TIME, IT NEVER OCCURRED TO ME THAT WE WOULD BE ABLE TO COME UP WITH SOMETHING LIKE WE DID TODAY. THIS IS ONLY POSSIBLE BECAUSE GOOGLE CLOUD AND TENSORFLOW COLLABORATES UNBELIEVABLY CLOSELY TOGETHER IN GOOGLE. TO BEGIN, LET’S TALK ABOUT TENSORFLOW. 46 MILLION DOWNLOADS. TENSORFLOW IS MASSIVE GROWTH THE LAST FEW YEARS. IT HAS EXPANDED FROM THE FOREFRONT OF RESEARCH WHICH WE HAVE SEEN EARLIER THIS MORNING TO BUSINESSES TAKING IT ON AS A DEPENDENCY FOR THEIR BUSINESS TO OPERATE ON A DAY IN, DAY OUT BASIS. IT IS JUST SUPER EXCITING PIECE AS SOMEONE WHO SPENDS MOST OF THEIR — MOST ALL OF THEIR TIME THINKING ABOUT HOW TO HAD BRING AI AND MACHINE LEARNING INTO BUSINESSES, SEEING TENSORFLOW’S COMMITMENT AND FOCUS ON DEPLOYING ACTUAL ML IN PRODUCTION, IT IS SUPER EXCITING TO ME. WITH THIS THOUGH, COMES GROWING PAINS. PART OF THAT, IT IS THINGS LIKE SUPPORT, MY MODEL DOESN’T DO WHAT I EXPECTED IT TO DO, MY TRAINING JOB FAILS, YOU KNOW, WHAT OPTIONS DO I HAVE? YOU KNOW, HOW WELL DOES YOUR BOSS RESPOND WHEN UP SAY, HEY, YES, I HAVE — I DON’T KNOW WHY MY MODEL IS NOT TRAINING, BUT NOT TO WORRY, I HAVE PUT A QUESTION ON SLACK AND HOPEFULLY SOMEONE WILL GET BACK TO ME. WE UNDERSTAND THAT BUSINESSES WHO ARE TAKING A BET ON TENSORFLOW AS A CRITICAL PIECE OF THEIR HARDWARE ARCHITECTURE, THEIR STACK, THEY NEED MORE THAN THIS. SECOND, IT CAN BE A CHALLENGE TO UNLOCK THE SCALE AND PERFORMANCE OF CLOUD. FOR THOSE OF YOU THAT LIKE ME HAVE GONE THROUGH THIS JOURNEY OVER THE LAST COUPLE OF YEARS, YOU KNOW, FOR ME IT STARTED ON MY LAPTOP, RIGHT, AND THEN EVENTUALLY I OUT GREW THE LAPTOP. SO I HAD A GAMING RIG UNDER MY DESK. EVENTUALLY THERE WERE EIGHT UNDER MY DESK, WHEN YOU OPENED THE DOOR TO THE OFFICE, THE WHOLE FLOOR KNEW, IT SOUNDED LIKE THIS, RIGHT! SO WITH TODAY’S CLOUD, THAT’S NOT THE CASE. YOU GO FROM THE SINGLE INSTANCE ALL THE WAY UP TO MASSIVE SCALE SEAMLESSLY. WITH THAT, TODAY, WE BRING YOU TENSORFLOW ENTERPRISE. THIS WILL GIVE YOU ENTERPRISE WIDE SUPPORT, TWO, CLOUD SCALE PERFORMANCE. THREE, MANAGED SERVICES WHEN AND WHERE YOU WANT THEM AT THE ABSTRACTION LEVEL YOU WANT THEM. ENTERPRISE GRADE SUPPORT, WHAT’S THAT MEAN? FUNDAMENTALLY WHAT IT MEANS, AS THE BUSINESSES TAKE A BET ON TENSORFLOW MANY OF THESE BUSINESSES HAVE IT POLICIES, REQUIREMENTS THAT THE SOFTWARE HUB, A CERTAIN LONGEVITY BEFORE THEY’RE WILLING TO COMMIT TO IT IN PRODUCTION. TODAY FOR CERTAIN VERSIONS OF TENSORFLOW WHEN USED ON GOOGLE CLOUD WE WILL EXTEND THAT ONE-YEAR SUPPORT IT A FULL THREE YEARS. THAT MEANS THAT IF YOU’RE BUILDING MODELS ON 1.15 TODAY, YOU CAN KNOW THAT FOR THE NEXT THREE YEARS YOU’LL GET BUG FIXES AND SECURITY PATCHES WHEN AND WHERE YOU NEED THEM. SIMPLE AND SCALABLE. SCALING FROM AN IDEA ON A SINGLE NODE TO PRODUCTION AT MASSIVE SCALE WITH CAN BE DAUNTING. YOU KNOW, SAYING TO MY BOSS, HEY, I TOOK A SAMPLE OF THE DATA, IT IS SOMETHING THAT PREVIOUSLY SEEMED TOTALLY REASONABLE, NOW WE’RE ASKED TO TRAIN ON THE ENTIRE CORPUS OF DATA, THAT CAN TAKE DAYS, WEEKS, WE COULD HELP WITH ALL OF HAD THAT BY DEPLOYING TENSE WERE FLOW ON GOOGLE CLOUD, A NETWORK THAT’S BEEN RUNNING TENSORFLOW SUCCESSFULLY FOR YEARS AND HAS BEEN HIGHLY OPTIMIZED FOR THIS PURPOSE. SO SCALABLE ACROSS OUR WORLD CLASS ARCHITECTURE, PRODUCTS ARE COMPATIBILITY TESTED WITH THE CLOUD, THEY’RE PERFORMANCED AND OPTIMIZED FOR THE CLOUD AND FOR GOOGLE’S WORLD CLASS INFRASTRUCTURE. WHAT’S THAT MEAN? IF ANY OF YOU HAVE EVER HAD THE OPPORTUNITY TO USE BIG QUERY, GOOGLE CLOUD’S MASSIVELY PARALLEL CLOUD HOSTED DATA WAREHOUSE, AND BY THE WAY, IF YOU HAVEN’T TRIED USING BIG QUERY, I HIGHLY RECOMMEND UP GO OUT, YOU TRY IT, IT IS — IT RETURNS RESULTS FASTER THAN CAN BE IMAGINED. THAT SPEED IN BIG QUERY, WE WANTED TO MAKE SURE THAT WE’RE TAKING FULL ADVANTAGE OF THAT. RECENT CHANGES, RECENT PIECES INCLUDED IN TENSORFLOW ENTERPRISE HAVE INCREASED THE SPEED OF THE CONNECTION BETWEEN THE DATA WAREHOUSE AND TENSORFLOW BY 3 TIMES. NOW ALL OF A SUDDEN, THE JOBS THAT WE’RE TAKING DAYS, THEY TAKE HOURS. UNITY GAMING, A WONDERFUL CUSTOMER AND PARTNER WITH US, YOU CAN SEE THE QUOTE HERE. UNITY LEVERAGES THESE ASPECTS OF TENSORFLOW ENTERPRISE IN THEIR BUSINESS. THE MODERNIZATION PRODUCTS REACH MORE THAN 3 BILLION DEVICES. 3 BILLION DEVICES WORLDWIDE. GAME DEVELOPERS RELY ON A MIX OF SCALE AND PRODUCTS TO DRIVE INSTALLS AND REVENUE AND PLAYER ENGAGEMENT. UNITY NEEDS TO BE ABLE TO QUICKLY TEST, BUILD, SCALE, DEPLOY, MODELS, ALL AT MASSIVE SCALE. THIS ALLOWS THEM TO SERVE UP THE BEST RESULTS FOR THEIR DEVELOPERS AND THEIR ADVERTISERS. MANAGED SERVICES, AS I HAVE SAID, TENSORFLOW ENTERPRISE WILL BE AVAILABLE ON GOOGLE CLOUD OF, AND WILL BE AVAILABLE AS PART OF GOOGLE CLOUD’S AI PLATFORM. IT WILL ALSO BE AVAILABLE IN VMS IF YOU PREFER THAT, OR IN CONTAINERS IF YOU WANT TO RUN THEM ON GOOGLE CLOUD KUPERNETES ENGINE OR USING KUPE FLOW ON THAT. IN SUMMARY, TENSORFLOW ENTERPRISE OFFERS ENTERPRISE GREAT SUPPORT, IT CONTINUES, THAT FULL THREE YEARS OF SUPPORT THAT IT DEPARTMENTS ARE ACCUSTOMED TO, CLOUD SCALE PERFORMANCE, SO THAT YOU CAN RUN AT MASSIVE SCALE AND WORKS SEAMLESSLY WITH OUR MANAGED SERVICES AND ALL OF THIS IS FREE AND FULLY INCLUDED FOR ALL OF GOOGLE CLOUD USERS. GOOGLE CLOUD BECOMES THE BEST PLACE TO RUN TENSORFLOW. THERE IS ONE LAST PIECE, WHICH IS FOR COMPANIES FOR WHOM AI IS THEIR BUSINESS, NOT COMPANIES FOR WHOM AI MAY HELP WITH THIS PART OF THEIR BUSINESS OR THAT OR MAY HELP OPTIMIZE THIS CAMPAIGN, THIS BACK END SYSTEM, BUT FOR COMPANIES WHERE AI IS THEIR BUSINESS, WHERE THEY’RE TRAINING HUNDREDS OF THOUSANDS OF HOURS OF TRAINING A YEAR, ALL OF THIS DATA, USING CUTTING EDGE MODELS, TO MEET THEIR UNIQUE REQUIREMENTS WE ARE INTRODUCING TENSORFLOW ENTERPRISE WITH WHITE GLOVE SUPPORT. THIS IS REALLY FOR CUTTING EDGE AI. ENGINEERING TO ENGINEERING ASSISTANCE WHEN NEEDED. YOU KNOW, CLOSE COLLABORATION ACROSS GOOGLE ALLOWS US TO FIX BUGS FASTER IF NEEDED. ONE OF THE GREAT OPPORTUNITIES OF WORKING IN CLOUD IF YOU ASK MY KIDS, THEY’LL TELL YOU THE REASON I WORK IN CLOUD AI AND IN MACHINE LEARNING, IT IS IN AN EFFORT TO KEEP THEM EVER FROM LEARNING TO DRIVE. THEY’RE 8 AND 10. I NEED PEOPLE TO KIND OF HURRY ALONG THIS ROUTE IF YOU WILL. ONE OF THE PARTNERS, ONE OF THE CUSTOMERS AND PARTNERS WE HAVE, CRUISE AUTOMOTIVE, AND YOU SEE HERE, THEY’RE A SHINING EXAMPLE OF THE WORK WE’RE DOING ON THIS — ON THEIR QUEST TOWARD SELF-DRIVING CARS. THEY HAVE EXPERIENCED HICK-UPS AND CHALLENGES AND SCALING PROBLEMS AND WE HAVE 9 BEEN A CRITICAL PARTNER FOR THEM IN HELPING ENSURE THAT THEY CAN ACHIEVE THE RESULTS THEY NEED TO SOLVE THIS KIND OF GENERATIONAL DEFINING PROBLEM OF YOU A TON MISS VEHICLES. — AUTONOMOUS VEHICLES. YOU CAN SEE. THAT NOT ONLY DO WE IMPROVE THE ACCURACY OF THE MODELS BUT REDUCED TRAINING TIMES FROM FOUR DAYS DOWN TO ONE DAY. THIS ALLOWS THEM TO ITERATE AT SPEEDS PREVIOUSLY UNTHINKABLE. NONE OF THIS, IT AS I SAID, WOULD HAVE BEEN POSSIBLE WITHOUT THE CLOSE COLLABORATION BETWEEN GOOGLE CLOUD AND TENSORFLOW. I LOOK BACK ON MEGAN’S RECENT ANNOUNCEMENTS OF THE BOA BOARD.DEV, WE’LL LOOK AT BRINGING THAT TYPE OF FUNCTIONALITY INTO AN ENTERPRISE ENVIRONMENT AS WELL IN THE COMING MONTHS, BUT WE’RE REALLY, REALLY EXCITED TO GET TENSORFLOW ENTERPRISE INTO YOUR HANDS TODAY. TO LEARN MORE, GET STARTED, YOU CAN GO TO THE LINK, AS WELL AS SESSIONS LATER TODAY. IF YOU ARE ON THE CUTTING EDGE OF AI, WE’RE ACCEPTING APPLICATIONS FOR THE WHITE GLOVE SERVICE AS WELL. WE’RE EXCITED TO BRING THIS OFFERING TO TEAMS, WE’RE EXCITED TO BRING THIS OFFERING TO BUSINESSES THAT WANT TO MOVE INTO A PLACE WHERE MACHINE LEARNING IS INCREASINGLY A PART OF HOW THEY CREATE VALUE. Y MUCH FOR YOUR- TIME TODAY. >>EDD WILDER-JAMES: THANK YOU, CRAIG. SO LEARN MORE ABOUT THE TECHNOLOGIES BEING COVERED AT THIS CONFERENCE, WE INVITE YOU TO START YOUR FREE 90-DAY MEMBERSHIP TO O’REILLY LEARNING, OUR ON-LINE PLATFORM OF EXPERT CONTENT AND TRAINING. IN ADDITION TO THOUSANDS FROM THE LEADING TECH PUBLISHER, THIS LEARNING PROGRAM HAS ON-LINE TRAININGS, AUDIO BOOKS, PLAY LISTS AND CONFERENCE VIDEOS FROM ALL O’REILLY EVENTS. WE’LL HAVE A PREMIUM COMPILATION OF TENSORFLOW WORLD VIDEO CONTENT. MAKE SURE YOU STOP AT THE O’REILLY BOOTH TO ACTIVATE YOUR FREE O’REILLY LEARNING MEMBERSHIP. OUR NEXT SPEAKER, KEMAL EL MOUJAHID, PRODUCT DIRECTOR FOR TENSORFLOW AT GOOGLE. HE’S PASSIONATE ABOUT SOLVING BIG PROBLEMS WITH AI AND DEVELOPEREVELOP WERE COMMUNITIES. WELCOME KEMAL EL MOUJAHID.>>KEMAL El MOUJAHID: WE HAVE A VERY LIVELY LEARNING COMMUNITY. IF THEY COME TO A ROAD SHOW, THEY’RE PART OF A COMMUNITY. THEY START TO UNDERSTAND THAT THEY CAN BE PART OF SOMETHING SO MUCH BIGGER.>>IT IS AN AMAZING PART OF THE ROAD SHOW, SEEING THE COMMUNITY IN NIGERIA, IT IS AMAZING. YES. TENSORFLOW, IT CHANGES — FOR ME, THE CONFERENCE LIKE THIS, IT IS TO ENSURE THE NETWORK, SEE WHAT ARE THE CHALLENGES AND SOLUTIONS ABOUT MACHINE LEARNING PROBLEMS. IT IS IMPORTANT TO WORK WITH THIS TOOL. IT IS MORE INTERESTING WHEN YOU SEE PEOPLE WORKING UNDERGROUND TO MAKE THIS, TO MAKE THIS TOOL AVAILABLE TO YOU. PART OF WHAT TENSORFLOW IS NOW, IT IS JUST THE ENGAGEMENT OF THE DEVELOPERS EVERYWHERE AROUND THE WORLD.>>TENSORFLOW, IT IS WITH THE COMMUNITY AND THE DIVERSE COMMUNITY RIGHT NOW, IT ORGANIZED EVENTS MONTHLY. THE BIGGER THE COMMUNITIES GETS, THE BETTER FOR US, THE BETTER FOR US, THE BETTER FOR THE COUNTRY.>>I LIKE TO BE A PART OF THE COMMUNITY. PEOPLE THAT ARE PASSIONATE ABOUT WHAT THEY’RE WORKING ON. THE WAY THAT THE COMMUNITY MOVES FORWARD, YOU ALSO MOVE FORWARD WITH THE COMMUNITY.>>I’M KEMAL EL MOUJAHID. I’M PRODUCT DIRECTOR FOR TENSORFLOW. EARLIER YOU HEARD FROM JEFF AND MEGAN ABOUT THE DIRECTION AND NOW I WOULD LIKE TO TALK ABOUT THE MOST IMPORTANT PART OF WHAT WE’RE BUILDING. THAT’S THE COMMUNITY. THAT’S — SORRY. THANK YOU. SO AS YOU HAVE SEEN IN THE VIDEO, WE HAVE HAD A GREAT ROAD SHOW. 11 EVENTS, SPANNING FIVE CONTINENTS, IT IS CONNECTING THE COMMUNITY WITH A TENSER NOPE TEAM. I PERSONALLY WAS VERY LUCKY THIS SUMMER, I GOT TO TRAVEL TO MOROCCO, TO GHANA, SHANGHAI, LONG OTHER PLACES, JUST TO MEET THE COMMUNITY AND TO LISTEN TO YOUR FEEDBACK. WE HEARD A LOT OF GREAT THINGS. SO AS WE’RE THINKING ABOUT HOW CAN WITH HE BEST HELP THE COMMUNITY, IT REALLY CAME DOWN TO THREE THINGS. FIRST, WE WOULD LIKE YOU TO HELP YOU CONNECT WITH THE LARGER COMMUNITY AND TO SHARE THE LATEST AND GREATEST OF WHAT YOU HAVE BEEN BUILDING. THEN WE ALSO WOULD LIKE YOU, WE WANT TO HELP YOU LEARN, LEARN ABOUT ML, LEARN ABOUT TENSORFLOW. THEN WE WANT TO HELP YOU CONTRIBUTE AND GIVE BACK TO THE COMMUNITY. LET’S START WITH CONNECT. SO WHY CONNECT? WELL, FIRST, THE COMMUNITY, THE TENSORFLOW COMMUNITY, THEY HAVE GROWN A LOT. IT IS HUGE, 46 MILLION DOWNLOADS, 2100 COMMITTERS, AND AGAIN I KNOW THAT WE HAVE BEEN SAYING THAT ALL ALONG, BUT I REALLY WANT TO SAY A HUGE THANK YOU ON BEHALF OF THE TENSORFLOW TEAM FOR MAKING THE COMMUNITY WHAT IT IS TODAY. ANOTHER ASPECT OF THE COMMUNITY THAT WE’RE VERY PROUD OF, IT IS THAT IT IS TRULY GLOBAL. THIS IS A REVISED MAP OF WHAT WE’RE COVERING, WE’RE COVERING ALL TIME ZONES AND WE’RE GROWING, THE COMMUNITY IS HUGE, IT IS TRULY GLOBAL, AND WE REALLY WANT TO THINK ABOUT YOU WHO CAN WE BRING THE COMMUNITY CLOSER TOGETHER AND THIS IS REALLY WHAT INITIATED THE IDEA OF TENSORFLOW WORLD. WE WANTED TO CREATE AN EVENT FOR YOU. WE WANTED AN EVENT WHERE YOU COULD COME UP AND CONNECT WITH THE REST OF THE COMMUNITY AND SHARE WHAT YOU HAVE BEEN WORKING ON. THIS HAS ACTUALLY STARTED ORGANICALLY. 7 MONTHS AGO, THE TENSE WERE FLOW USER GROUP STARTED, AND I THINK NOW WE HAVE CLOSE TO 50, THE LARGEST ONE, IT IS IN KOREA AND IT HAS 46,000 MEMBERS. WE HAVE 15 IN CHINA. IF YOU ARE IN THE AUDIENCE, IN A LIVE STREAM, YOU’RE LOOKING AT THIS MAP, SAYING WAIT, I DON’T SEE — I DON’T SEE A DO THE WHERE I LIVE, AND YOU HAVE A TENSORFLOW MEMBER THAT YOU’RE CONNECTING WITH, YOU WANT TO START A TENSORFLOW USER GROUP, WE WOULD LIKE TO HELP YOU. GO TO TENSER FLO FLOWER.ORG/COMMUNITY AND WE WILL GET YOU STARTED SO THAT NEXT YEAR WHEN WE LOOK AT THIS MAP, WE HAVE DOTS ALL OVER THE PLACE. WHAT ABOUT BUSINESSES? WE TALKED ABOUT DEVELOPER, LET’S TALK ABOUT BUSINESSES. ONE THING WE HEARD FROM BUSINESSES IS THEY HAVE THIS BUSINESS PROBLEM, THEY THINK ML CAN HELP THEM, THEY’RE NOT SURE HOW. THAT’S A HUGE MISSED OPPORTUNITY. WHEN WE LOOK AT THE STAGGERING 13 TRILLION-DOLLARS THAT AI WILL BRING TO THE GLOBAL ECONOMY OVER THE NEXT DECADE, SO YOU HAVE THOSE BUSINESES ON ONE SIDE, THEN YOU HAVE PARTNERS ON THE OTHER SIDE, THEY KNOW ML, KNOW HOW TO USE TENSORFLOW, HOW DO WE CONNECT THOSE TWO. THIS IS THE INSPIRATION FOR LAUNCHING OUR TRUSTED PARTNER PILOT PROGRAM WHICH HELPS THE BUSINESS CONNECT TO A PARTNER WHO WILL HELP YOU SOLVE YOUR ML PROBLEM. SO IF YOU GO ON TENSO TENSORFLOW.ORG AND YOU’LL LEARN MORE ABOUT THE TRUSTED PARTNER PROGRAM, A COUPLE OF EXAMPLES, SOME COOL THINGS THEY HAVE BEEN WORKING ON. ONE PARTNER HELPED A COMPANY SHORTEN THE INSURANCE CLAIM PROCESSING TIME USING IMAGE PROCESS TECHNIQUES. ANOTHER PARTNER HELPED A GLOBAL COMPANY BY AUTOMATING THE SHIPPING, LABELING PROCESS WITH OBJECT TECHNIQUE, YOU WILL LEARN MORE ABOUT THAT TODAY, I ENCOURAGE YOU TO CHECK OUT THE TALKS. ANOTHER ASPECT, IF YOU’RE A PARTNER, YOU ARE INTERESTED IN GETTING THIS PROGRAM, WE WOULD ALSO LIKE TO HEAR FROM YOU. 1S. LET’S TALK ABOUT LEARN. WE HAVE INVESTED A LOT ABOUT PRODUCING QUALITY MATERIAL TO HELP YOU LEARN ABOUT ML AND ABOUT TENSORFLOW. ONE THING WE DID OVER THE SUMMERS, IT WAS REALLY EXCITING, IT IS FOR THE FIRST TIME WE’RE A PART OF THE GOOGLE SUMMER OF CODE. WE HAD A LOT OF INTERESTS, WE WERE ABLE TO SELECT 20 VERY TALENTED STUDENTS, AND THEY GOT TO WORK THE WHOLE SUMMER WITH AMAZING MENTORS ON A TENSORFLOW ENGINEERING TEAM AND THEY WORKED ON VERY INSPIRING PROJECTS GOING FROM 2.0 TO SWIFT TO JS, TO TF AGENTS, SO WE WERE SO EXCITED WITH THE SUCCESS OF THE PROGRAM THAT WE DECIDED TO PARTICIPATE FOR THE FIRST TIME IN THE GOOGLE CODING PROGRAM. THIS IS THE SAME PROM BUT FOR PREUNIVERSITY STUDENTS FROM 1317, A GLOBAL ON-LINE CONTEST, AND IT INTRODUCED TEEN AGERS TO THE WORLD OF CONTRIBUTING TO OPEN SOURCE DEVELOPMENT. WE HAVE INVESTED A LOT ON ML EDUCATION MATERIAL. ONE THING WE HEARD, THERE IS A LOT OF DIFFERENT THINGS, AND WHAT YOU WANT, IT IS TO BE GUIDED THROUGH PATHWAYS OF LEARNING. SO WE HAVE WORKED HARD ON THAT, WE ARE EXCITED TO ANNOUNCE THE NEW LEARN ML PAGE ON TENSORFLOW.ORG. WHAT IT IS, IT IS A LEARNING PATH CREATED FOR YOU BY THE TENSORFLOW TEAM AND ORGANIZED BY LEVELS, YOU HAVE FROM BEGINNERS TO ADVANCE. YOU CAN EXPLORE BOOKS, COURSES, VIDEOS, HELP YOU IMPROVE YOUR KNOWLEDGE OF MACHINE LEARNING AND USE THAT KNOWLEDGE AND USE TENSORFLOW TO SOLVE YOUR REAL WORLD PROBLEMS. FOR MORE EXCITING NEWS, IT WILL BE AVAILABLE ON THE WEBSITE. I WOULD LIKE TO PLAY A HAD BRIEF VIDEO BY A FRIEND.>>HEY, EVERYONE. I’M IN NEW YORK NOW. I WISH I COULD BE THERE TO ENJOY THE CONFERNCE. I WANT TO SHARE WITH YOU SOME EXCITING UPDATES. DEEP LEARNING AI STARTED A PARTNERSHIP WITH THE TENSORFLOW TEAM WITH A GOAL OF MAKING WORLD CLASS EDUCATION AVAILABLE FOR DEVELOPERS ON THIS PLATFORM. SINCE RELEASING THE DEEP LEARNING SPECIALIZATION I HAVE SEEN SO MANY OF YOU, HUNDREDS OF THOUSANDS, LEARNING THE FUNDAMENTAL SKILLS OF DEEP LEARNING. I’M DELIGHTED WE HAVE BEEN ABLE TO COMPLEMENT THAT WITH THE TENSORFLOW IN PRACTICE SPECIALIZATION TO HELP DEVELOPERS LEARN HOW TO BUILD ML APPLICATIONS THROUGH COMPUTER VISION, SEQUENCE MODELS AND MORE. TODAY I WANT TO SHARE WITH YOU AN EXCITING NEW PROJECT THAT DEEP LEARNING.AI AND TENSORFLOW TEAMS HAVE BEEN WORKING ON TOGETHER. BEING ABLE TO USE YOUR MODELS IN A REAL WORLD SCENARIO IS WHEN MACHINE LEARNING GETS PARTICULARLY EXCITING. SO WE’RE PRODUCING A NEW FOUR-COURSE SPECIALIZATION, TENSORFLOW DATA AND DEPLOYMENT THAT WILL LET YOU TAKE THE SKILLS TO THE REAL WORLD, DEPLOYING MODELS TO THE WEB, MOBILE DEVICES AND MORE. IT WILL BE AVAILABLE IN EARLY DECEMBER. I’M EXCITED TO SEE WHAT YOU DO WITH THESE NEW RESOURCES. KEEP LEARNING.>>KEMAL El MOUJAHID: THIS IS REALLY COOL. SHINS WE STARTED TO WORK ON THE PROGRAMS, IT HAS BEEN AMAZING TO SEE HUNDREDS OF THOUSANDS OF PEOPLE TAKE THOSE COURSES AND THE GOAL OF THESE EDUCATIONAL RESOURCES IS TO LET EVERYONE PARTICIPATE IN THE ML REVOLUTION. REGARDLESS OF WHAT YOUR EXPERIENCE WITH MACHINE LEARNING IS. SO A GREAT WAY TO GET INVOLVED, ITS TO CONNECT WITH YOUR GE. WE NOW HAVE 126 MACHINE LEARNING GDES GLOBALLY, WE LOVE THEM, THEY’RE AMAZINGER, THEY DO AMAZING THINGS FOR THE COMMUNITIES. THIS YEAR ALONE, THEY GAVE OVER 400 TECH TALKS, 450 WORKSHOPS, THEY WROTE 221 ARTICLES, REACHING TENS OF THOUSANDS OF DEVELOPERS. ONE THING THAT WAS NEW THIS YEAR, IT IS THAT THEY HELPED WITH DOCS. YOU NEED REALLY GOOD QUALITY DOCS TO WORK ON MACHINE LEARNING AND OFTEN THE DOCUMENTATION IS NOT AVAILABLE IN PEOPLE’S NATIVE LANGUAGES. SO THIS IS WHY WE WHEN PARTNERING WITH OUR GDDS, WE LAUNCH THESE SPRINTS, IT WAS OVER 15 COUNTRIES, AND WE HEARD AMAZING STORIES OF THE POWER OUTAGE, POWER RUNNING OUT, PEOPLE COMING BACK LATER TO FINISH THE DOC SPRINTS AND WRITING THEM ON THEIR PHONES. IF YOU HAVE BEEN HELPING WITH DOCS, THANK YOU. IF YOU’RE IN THE ROOM, OVER LIVE STREAM, THANK YOU SO MUCH. IF YOU’RE INTERESTED IN HELPING TRANSLATE DOCUMENTATION IN YOUR NATIVE LANGUAGE, PLEASE REACH OUT AND WE’LL HELP YOU ORGANIZE A DOC SPRINTS. ANOTHER THING THAT IS HELPED WITH, EXPERIMENTING WITH THE LATEST FEATURES. I WANT TO CALL OUT ONE FROM SINGAPORE WHO HAS ALREADY EXPERIMENTED WITH 2.0 AND YOU CAN HEAR HIM TALK LATER TODAY TO HEAR ABOUT THE EXPERIENCE. SO IF YOU WANT TO GET INVOLVED, PLEASE REACH OUT AND START WORKING ON TENSORFLOW. ANOTHER REALLY GREAT WAY TO HELP, IT IS TO JOIN A SIG, A SPECIAL INTEREST GROUP, IT HELPS YOU TO WORK ON THE THINGS THAT YOU’RE THE MOST EXCITED ABOUT ON TENSORFLOW. WE HAVE NOW 11SIGS AVAILABLE. ADD I DON’T KNOWS, IO, NETWORKING IN PARTICULAR, IT REALLY SUPPORTED THE TRANSITION TO 2.0 BY EMBRACING THE PARTS OF CONTRIBUTING AND PUTTING THEM IN 2.0 AND THIS ENSURES THAT TF RUNS WELL EVERYWHERE ON ANY EO SECTION OR ARCHITECTURE AND PLAYS WELL WITH THE PYTHON LIBRARY. WE HAVE MANY OTHER REALLY EXCITING SIGS AND I ENCOURAGE YOU TO JOIN ONE. ANOTHER REALLY GREAT WAY TO CONTRIBUTE, IT IS THROUGH COMPETITION. FOR THOSE OF YOU WHO WERE THERE AT THE DEV SUMMIT BACK IN MARCH, WE LAUNCHED THE 2.0 CHALLENGE. THE GRAND PRIZE, IT WAS AN INVITATION TO THIS EVENT, TENSORFLOW WORLD. SO WE WOULD LIKE TO HONOR OUR 2.0 CHALLENGE WENTERS, AND I THINK WE’RE LUCKY TO HAVE TWO OF THEM IN THE ROOM, VICTOR, AND KYLE. IF YOU’RE HERE. VICTOR WORKED ON THE LIBRARY FOREHAND GESTURE IN THE BROWSER, AND THEN KYLE WORKED ON A PYTHON PACKAGE TO SIMULATE END BODY SIMULATIONS. SO ONE THING WE HEARD TOO DURING OUR TRAVELS, IT IS OH, THAT WAS GREAT, BUT I TOTALLY MISSED IT, CAN WE HAVE ANOTHER ONE? YES. LET’S DO ANOTHER ONE. IF YOU GO ON-LINE, WE’RE LAUNCHING A NEW CHALLENGE, YOU CAN APPLY YOUR 2.0 SKILLS AND SHARE THE LATEST AND GREATEST AND WIN COOL PRIZES. SO REALLY EXCITED TO SEE WHAT YOU’RE GOING TO BUILD. ANOTHER GREAT COMMUNITY THAT WE’RE EXCITED TO PARTNER WITH, WE’RE LAUNCHING CONTEXT TO CHALLENGE YOU WITH A QUESTION ANSWERING MODEL BASED ON AN ARTICLE. YOU CAN PUT YOUR NATIONAL LANGUAGE PROCESSING SKILLS TO THE TEST AND EARN 50,000 IN PRIZES. IT IS OPEN FOR ENTRY UNTIL JANUARY 22ND. SO BEST OF LUCK. WE HAVE A FEW ACTION ITEMS FOR YOU. THEY’RE LISTED ON THIS SLIDE. REMEMBER, WE CREATED TENSORFLOW WORLD FOR YOU. TO HELP YOU CONNECT AND SHARE WHAT YOU HAVE BEEN WORKING ON. OUR MAIN ACTION ITEM FOR YOU, IN THE NEXT TWO DAYS, IT IS REALLY TO GET TO KNOW THE COMMUNITY BETTER. WITH THAT, I WOULD LIKE TO THANK YOU AND I HOPE YOU ENJOY THE REST OF TF WORLD. THANK YOU.>>BEN LORICA: THANK YOU. THAT’S AWESOME! SO MANY WAYS TO GET INVOLVED. OUR THANKS TO IBM FOR SPONSORING THIS AFTERNOON’S BREAK IN THE EXPO HALL. PLAN TO JOAN US AT 5:45 P.M. TONIGHT FOR OUR RECEPTION. HAVE A DRINK, SOME DELICIOUS FOOD, AND NETWORK WITH OTHER TENSORFLOW WORLD ATTENDEES AND VISIT OUR SPONSORS WHO ARE LEADING THE CHARGE IN TENSORFLOW. NOW FROM 7:00 TO 10:00 P.M. TONIGHT, JOIN US AT CALIFORNIA’S GREAT AMERICA FOR THE OFFICIAL ATTENDEE PARTY, HALLOWEEN COSTUMES ARE ENCOURAGED BUT NOT REQUIRED. IT WILL BE A FUN EVENING FILLED WITH GREAT FOOD, DRINKS, AND A CHANCE TO RELAX WITH YOUR PEERS HERE AT TENSORFLOW WORLD. BE SURE TO BRING YOUR CONFERENCE BADGE TO GET IN. WE’LL BE MEETING OVER BY THE CONFERENCE REGISTRATION TO EITHER WALK OVER TOGETHER OR GET A PEDICAB. AND NOW, PLEASE JOIN US FOR OUR MORNING BREAK IN THE EXPO HALL. AFTER WARD, THE PROGRAM SPLITS INTO SESSIONS, CHECK OUT THE ON-LINE SCHEDULE FOR THE LATEST DETAILS. DON’T FORGET, IF YOU’RE ON THE LIVE STREAM, CORE SESSIONS WILL RESUME AFTER THE BREAK AND OTHERWSE JOIN US BRIGHT AND EARLY TOMORRO MORNING. WE HAVE GREAT DEMOS SET OUT FOR YOU. HAVE A GREAT DAY! A A OUR FIRST SPEAKER, JOSHODUCE GORDON ON THE TENSORFLOW TEAM, HE’LL TALK ABOUT THE EASE OF TENSORFLOW 2.0 AND WALK US THROUGH HE THE THREE STYLES OF MODEL APIS AND CODING EXAMPLES.>>JOSH GORDON: LET ME UNLOCK THIS LAPTOP AND WE’LL GET STARTED. I HAVE ONLY GOOD NEWS ON TENSORFLOW 2.0, IT IS A MONTH OLD. IT IS WONDERFUL, MASSIVELY EASIER TO USE AND GREAT FOR BEGINNERS AND FOR EXPERTS AND ALSO FROM A TEACHING PERSPECTIVE, IT HAS A LOT OF THINGS TO RECOMMEND THAT I’LL TALK ABOUT TOO. NO BIG DEAL, MAYBE WE CAN CLOSE THE DOORS IN THE BACK. IT IS A LITTLE LOUD. THANK YOU. I’LL GET INTO AN OUTLINE. ONE THING I WANT TO MENTION OFF THE BAT, IT IS TENSORFLOW 2 HAS ALL THE POWER OF GRAPHS THAT WE HAD IN TENSORFLOW ONE BUT THEY’RE HASIVELY EASIER TO USE. IN TENSORFLOW 1, THE NAME TENSORFLOW, TENSER IS BASICALLY A FANCY WORD FOR AN ARRAY, SCALERS, TENSERS, A CUBE IS A TENSER, AND FLOW, IT IS A DATAFLOW GRAPH. TENSORFLOW 1, YOU MANUALLY DEFINE A GRAPH AND EXECUTE IT WITH A SESSION. THIS FELT A LITTLE BIT LIKE META PROGRAMMING. THIS IS EXACTLY THE SYSTEM YOU WOULD WANT SEVERAL YEARS BACK IF YOU WERE AN ENGINEER AND THE CHALLENGE YOU FACED WAS MASSIVELY DISTRIBUTED TRAINING. AS A DEVELOPER, A STUDENT, A RAE SEARCHER, YOU WANT SOMETHING THAT FEELS MORE LIKE PYTHON. ON THE RIGHT, YOU SEE HOW TENSORFLOW 2 LOOKS. YOU THINK OF A TENSER IN A SIMILAR WAY TO AN INDIA ARRAY AND YOU WORK WITH THEM LIKE IN PYTHON. IT WORKS AS YOU WOULD EXPECT. IT IS GREAT. THAT’S LOW-LEVEL DETAILS. HERE ARE SOME THINGS I WOULD LIKE TO TALK ABOUT. THIS IS A ROUG ROUGH SOURPUSSS MATTOCK OF HOW THIS LOOKS, THIS IS MANY, MANY MOVING PIECES, IT IS A FRAMEWORK FOR MACHINE LEARNING. WHAT I WOULD LIKE TO DO, SHOW YOU A COUPLE OF THE PIECES AND WHAT SOME OF THE OPTIONS ARE TO USE IT. WE WILL START WITH DESIGNING MODELS USING MY FAVORITE API OF ALL TIME, AND WHAT YOU A SON IN 2, THIS IS REALLY AN IMPORTANT POINT, THERE IS A SPECTRUM OF USE CASES AND ALL OF THESE ARE BUILT INTO THE SAME FRAMEWORK AND YOU CAN MIX AND MATCH AS YOU GO. WHAT THIS MEANS, IT IS IF YOU’RE A TOTAL NOVICE TO DEEP LEARNING, YOU CAN START WITH SOMETHING CALLED THE SEQUENTIAL API, IT IS BY FAR THE EASIEST AND CLEAREST WAY TO DEVELOP DEEP LEARNING MODELS TODAY. IT IS WONDERFUL. YOU CAN BUILD A STACK OF LAYERS, YOU CAN COMPILE AND FIT AND THAT’S 100% VALID TENSORFLOW 2 CODE. IT IS JUST AS FAST AS ANY OTHER WAY OF WRITING CODE, BUT THERE IS NO DOWN SIDES TO IT TALL IF THE USE CASE FALLS INTO THE BUCKET. WHAT’S IMPORTANT IN TENSORFLOW 2, IT IS AS IS HELPFUL TO YOU, YOU CAN OPTIONALLY SCALE UP COMPLEXITY USING THINGS LIKE THE FUNCTIONAL API AND GOING ALL THE WAY TO SUB CLASSING HAUL IN THE SAME FRAMEWORK, YOU MIX AND MATCH AS YOU GO. WHAT THIS MEANS, WHEN I’M TEACHING THIS, FORKS EVER FOR EXAMPLE, YOU START ON THIS SIMPLE, CLEAR EASY WAYS AND WHEN I WANT TO WRITE FROM SCRATCH, I CAN DO THAT, WRITING CUSTOM LAYERS FROM SCRATCH, I CAN DO THAT, IT IS EASY. LET’S LOOK AT WHAT’S ON THESE PIECES AND ABOUT WHAT THEY LOOK LIKE. I HAVE ONE SLIDE ON SEQUENTIAL, WE HAVE SO MANY TUTORIALS ON IT, THERE IS A BOCK ON IT WHICH I’LL REBEL MEND AT THE END. I HAVE ONE SLIDE ON THIS. IN CASE YOU’RE NEW TO THIS, THE SEQUENTIAL API LET’S YOU DEFINE A STACK OF LAYERS AND THIS IS BY FAR THE MOST COMMON WAY TO MR. MODELS IN 880 TO 90% OF MACHINE LEARNING MODELS IF IT IN THIS FRAMEWORK. YOU FIND A STACK OF MODELS AND THAT’S GREAT. WHAT’S INTERESTING, WHEN UP USE THE SEQUENTIAL API AND THE FUNCTIONAL, A LOT OF DEVELOPERS DON’T REALIZE THIS, YOU’RE DEFINING A DATA STRUCTURE AND THIS MEANS THAT YOU CAN DO THINGS LIKE SUMMARY AND SEE A PRINT OUT OF THE LAYOUT AND WEIGHTS AND WE CAN DO COMPILE TIME CHECKS SO WHEN YOU CALL MOD MODEL.COMPILE YOU CAN MAKE SURE ALL OF THE LAYERS ARE COMPATIBLE. WHEN YOU SHARE YOUR MODEL 9WITH OTHER PEOPLE, IT MEANS FOR EXAMPLE IMAGINE THAT YOU WANT TO DO FINE TUNING, FOR TRANSFER LEARNING, IF YOU HAVE A MODEL THAT’S DEFINED WITH SEQUENTIAL API OR THE FUNCTION API, BECAUSE IT HAS A DATA STRUCTURE OF A STACK OF LAYERS OR THE FUNCTIONAL API AND A GRAPH OF LAYERS, YOU CAN INDEPENDENT THAT DATA STRUCTURE AND PULL LAYERS OUT OF IT AND GET THE ACTIVATIONS AND YOU CAN DO FINE TUNING, THINGS LIKE THAT EASILY. DEFINING STACKS OF LAYERS IS COMMON. IN TENSORFLOW 2 IT WORKS THE SAME WAY HERE. THAT’S GREAT. ANOTHER THING THAT’S POWERFUL IS THE SEQUENTIAL, THIS IS FOR BUILDING STACKS. I WANT TO SHOW YOU HOW POWERFUL THIS IS, TO BE HONEST, MOST OF WHAT I HAVE DONE MYSELF, USING THE SEQUENTIAL API OR GOING TO SUB CLASSING AND JUST WRITING EVERYTHING FROM SCRATCH. I HEARD AN AWESOME TALK ON THE FUNCTIONAL API IN MONTREAL AND I HAVE BEEN USING IT A LOT SENSE. I LOVE IT. I WANT TO SHOW YOU WHAT IT CAN DO. I WANT IT O SHOW YOU WHAT A QUICK MODEL LOOKS LIKE FOR VISUAL QUESTION ANSWERING. A LOT OF THE TIME WHEN YOU START WITH MACHINE LEARNING, IT IS A LOT OF TIME, DEEP LEARNING, YOU SPEND A LOT OF TIME BUILDING IMAGE CLASSIFIERS AND DOING THINGS LIKE CATS AND DOGS. WE CAN TAKE A LOOK AT A SLIGHTLY MORE SOPHISTICATED MODEL, A QA. THE DQA, YOU’RE GIVEN TWO INPUTS, ONE, YOU’RE GIVEN AN IMAGE. HERE WE HAVE THE PAIR OF DOGS AND GIVEN A QUESTION IN AN ACTUAL LANGUAGE. HERE THE QUESTIONS ARE ASKING WHAT COLOR IS THE DOG ON THE RIGHT? TO ANSWER A QUESTION LIKE THIS, YOU NEED A MUCH, MUCH, MUCH MORE SOPHISTICATED MODEL THAN AN IMAGE CLASSIFIER. YOU CAN PHRASE THIS AS A CLASSIFICATION PROBLEM. IF YOU GO HERE, THERE ARE TWO EXCELLENT PAPERS THAT WILL GO IN DETAIL. YOU CAN IMAGINE YOU HAVE SOME MODEL, BULLETS TALK ABOUT HOW WE DO THIS. HERE WE HAVE A MODEL WITH TWO INPUTS, A IMAGE AND QUESTION. IF YOU TAKE A MACHINE LEARNING COURSE, DEEP LEARNING COURSE YOU LEARN ABOUT THE PROCESSING IMAGES WITH LAYERS AND YOU LEARN ABOUT TEXT AND EMBEDDING, A THING THAT’S REALLY POWERFUL ON DEEP LEARNING, IT IS ALL OF THESE LAYERS, REGARDLESS OF WHAT THEY ARE, THEY TAKE VECTORS AS INPUT. IF YOU’RE A DENSE LAYER, YOU DON’T CARE IF YOUR INPUT HAPPENS TO BE THE OUTPUT OF SOME OTHER LAYER OR IF THE INPUT HAPPENS TO BE THE OUTPUT OF AN LSTM. IT IS NUMBERS THAT YOU’RE TAKING AS INPUT. IN DEEP LEARNING, THERE IS NO REASON WE CAN’T PROCESS THE IMAGE WITH A CNN, PROCESS THE TEXT WITH AN LSTM AND THEN LOOK AT THE RESULT AND FEED THAT INTO A DENSE LAYER AND PHRASE THIS AS A CLASSIFICATION ROB. YOU CAN IMAGINE THE OUTPUT OF THE DEPS LAYER, 1,000 DIFFERENT CLASSES AND EACH CLASS CORRESPONDS TO ONE POSSIBLE ANSWER. HERE IF THE ANSWER IS GOLDEN, WE WANT TO CLASSIFY BOTH INPUTS JOINTLY AS GOLDEN. I WANT TO SHOW YOU HOW QUICKLY WE CAN DESIGN SOMETHING LIKE THIS WITH THE FUNCTIONAL API WHICH IS AMAZING. SO COOL. I PROBABLY SHOULD HAVE FLIPPED TO THIS SLIDE BUT THIS IS WHAT I’M TALKING THROUGH. THIS IS THE ARCHITECTURE WE WANT. WE WANT — THIS IS ONE MODEL, AND IT WILL HAVE TWO HEADS, 9AND THE FIRST HEAD, IT WILL BE A STANDARD STACK OF CNNS AND MAX PULLING LAYERS AND THIS IS EXACTLY THE SAME MODEL YOU USE FOR CLASSIFYING CATS AND DOGS. YOU CAN DO ALL OF THE SAME TRICKS YOU LEARN ABOUT THERE. YOU CAN IMPORT LIKE HERE, I WANT TO SHOW HOW TO WRITE IT FROM SCRATCH BUT YOU SHOULDN’T BE ABLE TO IMPORT THIS AND USE THAT TO GET ACTIVATIONS FOR THE IMAGES. BASICALLY WE GO FROM AN IMAGE TO A VECTOR. AND IN THE OTHER HEAD, WE’LL PROCESS THE QUESTION. WE’LL GO FROM THE QUESTION TO A VECTOR. WITH THAT, WE USE THE EMBEDDING AND THE LSTM AND THEN WE CAN LOOK AT THE RESULTS AND CLASSIFY IT. THIS IS NEARLY THE COMPLETE CODE FOR THE ENTIRE MODEL. IT IS THE COMPLETE CODE FOR THE MODEL, THE VERSION OF IT WHICH IS NUTS WHEN YOU LOOK AT IT. HERE, THIS IS OUR IMAGE CLASSIFIER. ING YOU WANTS SOMETHING DEEPER BUT THIS IS THE HELLO WORLD IMAGE CLASS FEAR. A VECTOR GOES IN, JUST SOME RANDOM TIPS, YOU SEE THE SLIDE, THAT’S COOL. YOU NOTICE IN THE FIRST LAYER, THIS IS A LAYER WITH 64 FILTER IT’S, 3 BY 3 WITH ACTIVATION AND IN THE INPUT SHAPE THERE, I’M SPECIFYING 9 HOW LARGE THE IMAGE IS. YOU WILL FIND THAT OFTEN THEY INFER THE INPUT SHAPE, WHENEVER YOU CAN, YOU SHOULD SPECIFY IT, IT IS ONE LESS THING THAT COULD GO WRONG. FULLY SPECIFY IT, CATCH BUGS EARLY. AFTER THAT, WE’RE DOING MAX PULLING AND THE IMPORTANT PART, WE’RE FLATTENING IT. THAT’S A SEQUENTIAL MODEL INSIDE OF THE FUNCTIONAL API. AFTER THAT, WE CREATE AN INPUT LAYER FOR THE FUNCTIONAL API AND BEGINNING TO CHAIN LAYERS TOGETHER TO BUILD UP A GRAPH. HERE WHAT WE’RE DOING, IT IS WE’RE CHAINING THE VISION MODEL TO THE INPUT LAYER. SO THAT’S THE FIRST HALF OF OUR MODEL. HERE IS THE SECOND HALF. WE’RE ALMOST DONE. THIS IS THE MODEL PROCESSING THE QUESTION. WE’RE CREATING ANOTHER INPUT. WE DON’T HAVE THE PREPROCESSING HERE. YOU CAN IMAGINE THAT WE TOKENIZE THE TEXT, VECTORIZED IT, PADDED IT, FED THAT INTO AN EMBEDDING AND THEN INTO AN LSTM. THIS IS EXACTLY WHAT YOU MAY DO HERE WITH THE TEXT CLASSIFIER. THE IMPORTANT THING, A VECTOR IS COMING UP, WE’RE CHANGE THESE HAS TO TOGETHER TO BUILD A HAD GRAPH. AT THE VERY END, THIS IS THE MAGIC BIT ABOUT DEEP LEARNING, WE SIMPLY LOOK AT THE RESULTS, NOTHING SIMPLE, AND IT IS ONE LINE OF CODE WHICH IS NICE. NOTHING SIMPLE CONCEPTUALLY. WHAT WE CAN DO, WE CAN LOOK AT THE RESULTS AND NOW WE HAVE A VECTOR. JUST LIKE ANY OTHER PROBLEM, NOW THAT WE HAVE THIS VECTOR, WE CAN FEED IT INTO DENSE LAYERS AND CLASSIFY IT. HERE IS THE TAIL OF THE 9 MODEL. NOW WE HAVE A TENSORFLOW 2 MODEL DOING THE VQA. THIS WILL WORK EXACTLY LIKE ANY OTHER MODEL. IF YOU WANT, YOU CAN CALL MODEL.FIT, MODEL ON TRAIN ON BATCH, CALL BACKS, IF YOU WANT, YOU CAN WRITE A CUSP TO TRAINING WITH THESE TAPES. SO I THINK THIS IS POWERFUL. WHAT’S NEXT, ABOUT THE FUNCTIONAL APIS, LIKE SEQUENTIALAL, THERE IS A DATA STRUCTURE BEHIND THE SCENE, THERE IS A GRAPH, 2 RUNS COMPATIBILITY CHECKS TO MAKE SURE LAYERS WORK WITH ONE ANOTHER. IT IS REALLY NICE. BASICALLY, IF YOU HAVEN’T USED THE FUNCTIONAL API, YOU JUST LEARNED ABOUT THE SEQUENTIAL FROM BOOKS, YOU’RE COMING FROM PYE TORCH AND YOU USED SUB CLASSING, I ENCOURAGE YOU TO TRY IT OUT. I HAVE HAD ONLY POSITIVE RESULTS IN THE LAST FEW WEEKS. I LOVE IT. OTHER THINGS, OF COURSE, SO THAT GRAPH I JUST MADE IN GOOGLE SLIDES, BUT BECAUSE YOU HAVE A DATA STRUCTURE, RATHER THAN CALLING SOMETHING LIKE MODEL.SUMMARY, YOU CAN PLOT IT AND GET A NICE RENDERING OF A GRAPH LOOKING EXACTLY WHAT I JUST SHOWED YOU. FOR COMPLICATED MODELS, THIS IS COOL, THE ONLY TIME I FOUND IT REALLY USEFUL, WHEN YOU HAVE COMPLICATED MODELS, THINGS LIKE RESNETS WHERE YOU PLOT OUT THE GRAPH AND IT LOOKS LIKE YOU EXPECT AS YOU ASSEMBLE IT. IT IS REALLY NICE. THEN THERE IS ANOTHER STYLE IN TENSORFLOW 2, THE LAST THING I SHOWED YOU, YOU CAN FIND THAT IN THE BACK END, AND IN TENSORFLOW 2, IT IS A SUPER SET OF WHAT YOU HAVE FOUND BEFORE, THIS IS NEW. THIS IS SUB CLASSING AND THIS IS A DEVELOPING STYLE MODEL WHICH IS REALLY, REALLY NICE AND YOU SEE INCREASING CONTROL AS YOU MOVE UP. YOU CAN DO A RESEARCHER, A STUDENT, THE FIRST TIME, WRITING EVERYTHING FROM SCRATCH TO LEARN HOW IT WORKS AND BECAUSE IT HAS SPECIAL OTHER ONES. HERE WHAT WE’RE DOING, WE’RE DEFINING A SUB CLASS MODEL. I REALLY LOVE THIS. THIS FEELS LIKE OBJECT ORIENTED DEVELOPMENT. WHAT WE’RE DOING, AND THE IDEA, ITS BASICALLY, IT IS VERY, VERY SIMILAR IN ALL OF THESE FRAMEWORKS. THE FRAMEWORK GIVES YOU A CLASS, HERE THIS CLASS HAPPENS TO BE MODELED AND THERE ARE TWO CHUNKS, YOU HAVE A CONSTRUCTSER AND CALL METHOD OR A FORWARD METHOD OR PREDICT METHOD. IN THE CONSTRUCTER, YOU DEFINE THE LAYERS. HERE I’M CREATING DENSE LAYERS AND IT IS THE EXACT SAME LAYERS THAT YOU FIND IN THE SEQUENTIAL AND FUNCTIONAL MODEL APIS WHICH IS GREAT. BASICALLY YOU LEARN THESE LAYERS ONCE, YOU USE THEM ALL OVER THE PLACE. YOU DESCRIBE HOW THE METHODS AND LAYERS ARE CHAINED TOGETHER, I FEED THE INPUT THROUGH THE DENSE LAYER AND FEEDING THAT RESULT THROUGH MY SECOND DENSE LAYER AND RETURNING IT. IF YOU’RE CURIOUS ON WHAT X IS, YOU DO PRINT X. THAT GIVES YOU THE ACTIVATIONS OF THE DENSE LAYER, IF YOU WANT, YOU WANT TO HIGHLIGHT IT. LET’S SAY FOR SOME REASON I’M NOT INTERESTED IN THE BUILT-IN RADIO ACTIVATION, I WANT TO WRITE MY OWN. I REMOVE THAT, YOU WAIT YOUR OWN WITH REGULAR PYTHON FLOW RIGHT THERE SO THEY’RE WRITTEN WITH THE BUILT-IN METHOD AND I CAN USE THAT WITH REGULAR PYTHON. THIS IS GREAT FOR IF YOU WANT TO REALLY KNOW THE DETAILS OF WHAT EXACTLY IS FLOWING IN AND OUT OF THE LAYERS. IT IS A PERFECT WAY OF DOING THAT. IT IS ALSO FOR EXAMPLE, THERE IS NOTHING HERE SAYING THAT YOU HAVE TO USE THE BUILD-IN DENSE LAYERS. IF YOU LOOK AT THE CODE FOR YOU ARE THE DENSE LEAR, THAT’S DOING SOMETHING LIKE THIS, YOU CAN ABSOLUTELY JUST WRITE THAT FROM SCRATCH AND USE THAT HERE INSTEAD. THERE IS A BUNCH OF REFERENCES. I’LL TWEET OUT THE SLIDES SO YOU DON’T HAVE TO WRITE IT DOWN. WHAT THESE ARE, THESE ARE DECIDES GUIDES THAT WILL GO INTO DETAIL ON HOW YOU WRITE HOW TO USE EACH OF THE THREE STYLES AND HOW MUCH DO YOU WRITE CUSTOM LAYERS, STUFF% LIKE THAT, THEY’RE GREAT. THEN WE HAVE A COUPLE OF TUTORIALS THAT I RECOMMEND THAT ARE NICE THAT SHOWS DIFFERENT WAYS OF BUILDING STUFF. IF YOU HAVE NOT SEEN THIS BY THE WAY, THE SEGMENTATION ONE, IT IS NICE, WE WROTE IT THIS SUMMER, IT RUNS REALLY, REALLY FAST, I THINK YOU’LL LIKE IT. ALSO BY THE WAY, BECAUSE THIS IS AN INTRO TO TENSORFLOW 2 TALK, LET ME SHOW YOU WHAT THE TUTORIALS ARE IN CASE YOU’RE INTO KNEW. A THING I WANT TO POINT OUT, THIS IS A TUTORIAL ON A WEBSITE. BIG SURPRISE. WHAT I WANTED TO MENTION, I THINK THIS IS A NICE FEATURE OF TENSORFLOW.ORG, THIS IS A WEBPAGE, HTML. THIS IS A DIRECT RENDERING OF THIS JUPITER NOTEBOOK. THE WEBPAGE IS THE NOTEBOOK. THE REASON WE HAVE DONE THAT, IT IS ALL THE TUTORIALS, THEY’RE RUNNABLE END-TO-END. SO YOU CAN INSTALL TENSORFLOW 2 LOCALLY AND RUN THIS TUTORIAL, THIS IS THE EXACT SAME PAGE YOU HAVE ON TENSORFLOW.ORG. FOR ALL OF THESE, YOU CAN DO DO ONE-TIME RUN ALL AND THIS HAS THE COMPLETE CODE TO REPRO DIES THE RESULTS THAT YOU SEE HERE. WHAT THIS MEANS, THE TUTORIALS ARE TESTABLE. YOU HAVE HEARD THE EXPRESSION OF TRUST BUT VERIFIED? FOR A LONG TIME I HAVE SEEN NIFTY TUTORIALS THAT ARE LIKE LET ME SHOW YOU HOW TO WRITE THIS NEURAL MACHINE MODEL AND THE CODE DOESN’T WORK OR MISSING A KEY PIECE THAT’S LEFT AS AN EXERCISE TO THE READER. AT LEAST ALL OF THE TUTORIALS ON THE WEBSITE, ALL RUN END-TO-END, IT IS REALLY, REALLY NICE. SOME WE STILL HAVE PLENTY OF WORK TO DO BUT THEY HAVE THE COMPLETE CODE TO DO THE THING. I REALLY LIKE THAT A LOT. I WANT TO TALK ABOUT TRAINING MODELS. SO BASICALLY THERE ARE SEVERAL WAYS TO TRAIN MODELS AND AGAIN YOU CAN USE — THE NICE THING ON TENSORFLOW 2, YOU USE THE ONE THAT’S MOST HELPFUL FOR THE USE CASE. YOU DON’T ALWAYS NEED TO WRITE THE CUSTOM TRAINING FROM SCRATCH SO YOU HAVE OTHER OPTIONS. THE FIRST IS THAT YOU MAY BE FAMILIAR WITH THIS, YOU CALL MODEL.FIT. WHAT’S NICE ABOUT MODEL.FIT, THIS DOESN’T CARE IF YOU HAVE THE SEQUENTIAL MODEL, FUNCTIONAL, SUB CLASS MODEL. IT WORKS FOR ALL OF THEM. MODEL.FIT, IT IS FAST, PERFORMING, SIMPLE. ONE THING THAT’S A LITTLE BIT LESS OBVIOUS, WHEN YOU MOD MODEL.FIT, THIS IS NOT JUST THE BABY WAY OF TRAINING MODELS. IF YOU’RE WORKING IN A TEAM, AND YOU CALL MODEL.FIT, YOU REDUCED YOUR CODE FOOTPRINT BY A LOT. THIS IS ONE LESS THING YOUR FRIENDS NEED TO WORRY ABOUT WHEN PLAYING WITH MODELS DOWN THE ROAD. IF YOU USE THE SIMPLE THINGS, YOU SHOULD. UNLESS THERE IS A REASON FOR MORE COMPLEXITY, OF COURSE, LIKE IN REGULAR SOFTWARE ENGINEERING. THE NICE THING ABOUT FIT, YOU PASS IN DIFFERENT METRICS AND EXAMPLES, YOU SEE ACCURACY, BY THE WAY, TENSORFLOW 2 HAS NICE METRICS FOR PRECISION AND RECALL, AND YOU WRITE CUSTOM METRICS AND SOMETHING THAT’S HELPFUL TOO, IT IS CALL BACKS. FOR INSTANCE, THESE ARE THINGS THAT I DON’T SEE A LOT OF NEW DEVELOPERS USING AND THEY’RE SUPER HELPFUL. CALL BACKS, ONE OF MY FAVORITES, EARLY STOPPING. TYPICALLY WHEN WE’RE TRAINING MODELS, YOU NEED TO PREVENT OVER FITTING AND A WONDERFUL WAY TO DO THAT, YOU MAKE PLOTS OF THE LOSS OVER TIME AND SO ON, SO FORTH, THIS DOES THAT AUTOMATICALLY WHICH IS HELPFUL AS WELL. YOU CAN WRITE CUSTOM CALL BACKS. A COOL THING, LET’S SAY YOU’RE TAKING YOUR TRAINING A MODEL THAT TAKES VERY LONG TIME TO TRAIN, YOU CAN WRITE A CALL BACK TO SEND YOU A SLACK NOTIFICATION AFTER EVERY TRAINING COMPLETES. THAT COULD BE REALLY NIFTY TOO. CALL BACKS, THEY’RE GREAT. I DON’T HAVE SLIDES FOR TRAIN ON BATCH HERE, I WANTED TO SHOW YOU A CUSTOM TRAINING WITH A GRADIENT TAPE. THIS IS ALSO VERY POWERFUL. ESPECIALLY IF YOU’RE A STUDENT LEARNING THIS FOR THE FIRST TIME AND DON’T WANTS A BLACK BOX AND FOR RESEARCHERS, HERE IS A CUSTOM TRAINING LOOP, I HAVE AN EXAMPLE FOR THIS IN A MINUTE, YOU CAN SEE EXACTLY HOW THIS WORKS. THIS IS A CUSTOM TRAINING LOOP. WHAT WE’RE DOING HERE, WE HAVE SOME FUNCTION, AND FOR NOW YOU PRETEND THAT THE FUNCTION IN THE ORANGE BOX, IT DOESN’T EXIST. THAT’S OPTIONAL. PRETEND IT DOESN’T EXIST. WE HAVE FUNCTION TAKING FEATURES AND LABELS AS INPUT. WHENEVER WE DO TRAINING AND DEEP LEARNING, WE’RE DOING DISSENT. THE FIRST STEP IN THIS, IT IS GETTING THE GRADIENTS AND THE WAY ALL FRAMEWORKS DO THIS, IT IS BY BACKDROP, REVERSE MODE AUTO DIP AND THE IMPLEMENTATION IN TENSORFLOW, WE START ECORDING OPERATIONS ON A TAPE. WE CREATE A TAPE AND RECORD WHAT’S HAPPENING BENEATH THAT TAPE WITH REGULAR CODE CALLING THE TOER WARD METHOD, THE CALL METHOD ON THE MODEL. WE FORWARD THE FEATURE THROUGH THE MODEL AND COMPUTING SOME LOSS AND MAYBE DOING REGRESSION IN THAT SQUARED ANTERIOR, WHAT WE’RE DOING, WE’RE GETTING THE GRADIENTS OF THE LOSS WITH RESPECT TO THE VARIABLES IN THE MODEL. IF YOU PRINT THEM OUT, WE’LL SEE WHAT THE GRADIENTS ARE AND HERE WE’RE DOING THE GRADIENT DISSENT MANUALLY, APPLYING THEM ON OPTIMIZER AND WE WRITE OUR OWN OPTIMIZER AND WE’LL DO THAT IN A SECOND. ANYWAY, THIS IS A CUSS BOMB TRAINING LOOP FROM SCRATCH. WHAT IT MEANS, IT IS THAT IF IN MODEL.FIT, YOU USE ALL OF THIS STUFF, BUT IF YOU WOULD LIKE TO WRITE THE OPTIMIZER, YOU GO AHEAD, YOU WRITE IT IN PYTHON AND THEY FIT IN WITH THE MODEL. IT IS GREAT FOR RESEARCH. TF FUNCTION BY THE WAY, FIRST OF ALL, YOU NEVER NEED TO WRITE IT, THE CODE WORKS THE SAME. IF YOU DO WANT A GRAPH IN TENSORFLOW 2, YOU BASICALLY WANT TO COMPILE THE CODE AND HAVE IT RUN FASTER, YOU WRITE THAT FUNCTION ANNOTATION AND WHAT THIS MEANS IS THAT TENSORFLOW 2 TRACES THE COMPUTATION, COMPILING IT, THE SECOND AND ON TIME THAT YOU RUN THIS FUNCTION, IT IS 9 MUCH, MUCH FASTER. IT IS RUNNING ENTIRELY IN C + +. ALL THERE IS TO GRAPHS IN TENSORFLOW 2 IS THE OPTION, IT IS OPTIONAL, YOU DON’T HAVE TO USE IT, IT IS EASY PERFORMANCE IF YOU NEED TO USE IT.>>I JUST WANT TO MAKE IT SUPER CONCRETE BECAUSE THIS IS GETTING STARTED WITH TENSORFLOW 2 TALK. AND THERE IS A LOT OF TUTORIALS THAT WILL QUICKLY SHOW ON THE WEBSITE HOW TO TRAIN IMAGE CLASSIFIERS, WHATNOT. A GOOD PLACE TO START TOO, IT IS LOOKING AT LINEAR REGRESSION AND THE REASON IS, IT IS GRADE I CAN’T WANT DISSENT, ALL ARE TRAINED BY THIS, AND A NICE PLACE TO START IS SEEING EXACTLY WHAT THIS IS. BECAUSE I HAVE — IT IS TINY, I HAVE A GRAPHIC ON THE LEFT. I’M JUST BRIEFLY GOING TO EXPLAIN HOW THE REGRESSION WORKS AND IT IS THE SAME PATTERN FOR DEEP NEURAL NETWORKS TO, WHICH IS SURPRISING. IN DEEP NEURAL NETWORKS, YOU SEE THREE THINGS, THE FIRST THING YOU NEED, A MODEL, WHICH IS A PREDICTION, AD MOAL, IT COULD BE Y EQUALS M PLUS B, WE’RE TRYING TO FIND THE BEST FIT LINE AND WE CAN DEFINE THE LINE, Y EQUALS THIS AND THEN WE HAVE TWO PARAMETERS FOR VARIABLES THAT WE NEED TO SET. WE HAVE M, WHICH IS THE SLOPE, RIGHT, AND THEN WE HAVE B, THE INTERSECT. BY WIGGLING THE VARIABLES, WE FIT THE LINE TO THE DATA. ON THE RIGHT YOU WILL SEE A PLOT, WE HAVE A SCATTERED PLOT WITH A BUNCH OF POINTS AND WE HAVE THE BEST LINE. THE IDEA, IT IS NOW THAT WE HAVE A MODEL, A LINE THAT WE CAN WIGGLE, WE NEED A WAY OF SAYING OR QUANTIFYING HOW WELL DOES THIS LINE FIT IN THE DATA. A WAY TO QUANTIFY HOW THE LINE FITS THE DATA, IT IS SQUARED AIR. YOU DROP THE LINE ON THE PAGE. THEN YOU MEASURE THE DISTANCE FROM THE LINE TO ALL OF THE POINTS. OF THAT. THE HIGHER THE SUM OF THE SQUARES IS, THE FURTHER — THE BETTER THE WAY THAT IT FITS THE DATA, THE LOWER THE SUM OF SQUARES. YOU HAVE A SINGLE NUMBER, IT DESCRIBES HOW BADLY THE LINE FITS THE DATA. THEN YOU WANT TO REDUCE THAT LOSS AND YOU KNOW THAT IF THE LOSS IS AT A MINIMUM, THE LINE FITS THE DATA WELL AND YOU FOUND THE BEST LINE. THE WAY WE REDUCE THAT LOSS IS GRADIENT DISSENT AND ON THE LEFT, YOU SEE THAT PLOT. WE’RE LOOKING AT LOSS OR A SQUARED ERROR AS A FUNCTION OF TWO VARIABLES, MM BECOMES. YOU SEE THAT IF WE SET THIS WITH A RANDOM GUESS TO START, A LOSS, IT IS HIGH. AND AS WE WIGGLE THEM, WE REDUCE THAT LOSS. THE TRICK, IT IS HOW DO WE FIGURE OUT WHICH WAY TO WIGGLE THIS? THERE ARE TWO WAYS TO DO THIS. IF YOU FORGET CALCULUS, YOU CAN FIND IT NUMERICALLY, IT IS NOT ROCKET SCIENCE. YOU TAKE M. WIGGLE IT, COMPUTE THE LOSS. YOU RECOMPUTE THE LOSS. THAT’S THE DIRECTION YOU’RE MOVING IN. BASICALLY, YOU DON’T HAVE TO USE THIS TALL, YOU CAN USE TENSORFLOW 2 MUCH LIKE THIS, WHENEVER YOU SEE SOMETHING LIKE THIS, YOU JUST REPLACE THAT AND YOU HAVE THE CONSTANT, IT IS PRINTED OUT WITH A SHAPE, A DATA TYPE, ONE NICE THING ABOUT TENSORFLOW TENSORS, THEY HAVE THIS METHOD THAT YOU GO STRAIGHT FROM TENSORS TO THIS, IT IS GREAT, YOU’RE FREE OF THE CLUTCHES OF TENSORFLOW 2. TENSOR HAS A SHAPE, A DATA SITE AND THEN JUST LIKE YOU WOULD EXPECT WITH THINGS LIKE DISTRIBUTION, IF YOU WANT TO CREATE RANDOM ROLES, THIS IS HOW YOU DO IT. I’LL DO IT QUICK. YOU CAN DO MATH IN TENSORFLOW 2 LIKE YOU DO MATH HERE. JUST LIKE YOU HAVE THINGS LIKE THE SQUARE AND MULTIPLIER, TENSORFLOW HAS ALL OF THIS TOO. THE IDEA IS THE SAME. YOU MAY HAVE TO POKE AROUND BUT THE NAMES, THEY’RE THE SAME, THEY’RE ALL THERE. THIS IS AN EXAMPLE OF VERY, VERY SIMPLE, THIS IS HOW WE GET THE GRADIENTS WITH THE GRADIENT TAPE AND THIS IS CONCRETE. WE HAVE A CONSTANT THAT’S 3. WE HAVE A FUNCTION WHICH IS X SQUARED. WHEN YOU THINK ABOUT THE RULES OF CALCULUS, IF YOU HAVE 3X SQUARED, YOU TAKE THE 2, YOU MULTIPLY IT BY THE 3, YOU GET 6. IF YOU WALK THROUGH THE CODE, YOU SEE IT RETURNS YOU 6-2. BASICALLY THIS IS HOW YOU GET THE GRADIENT TAPE. YOU GET THAT WITH THE VARIABLES AND LAYERS AT ONCE. YOU HAVE A PAIR OF DENSE LAYERS AND WE’RE CALLING THE DENSE LAYERS ON THE DATA AND GETTING THE GRADIENTS ALSO UNDER THE TAPE AND LET ME SHOW YOU WHAT THIS LOOKS LIKE TO MAKE THIS CONCRETE. THIS IS CODE FOR Y EQUALS MX PLUS B. THE FIRST THING TO MENTION IS HOW DO YOU INSTALL TENSORFLOW 2. IF YOU’RE RUNNING IN COLAB, RIGHT NOW IN COLAB, TENSOR 1 IS INSTALLED BY DEFAULT. THERE IS A MAGIC FLAG TO RUN, THAT MAGIC COMMAND GIVES YOU THE VERSION OF TENSORFLOW 2. IF YOU VISIT US LOCALLY, YOU CAN DO AN INSTALL, ANYTHING YOU WANT TO DO, YOU CAN DO IT LOCAL, I HAVE IT HERE BECAUSE IT IS CONVENIENT. TENSORFLOW IS JUST A PYTHON LIBRARY. YOU CAN IMPORT IT AS YOU ALWAYS WOULD. THE FIRST THING WE DO IN THIS NOTEBOOK, I KNOW I WENT THROUGH THE CODE EXAMPLES, BUT WHAT WE’LL DO, WE’LL CREATE A SCATTERPLOT. RANDOM DATA. WE’LL FIND THE BEST FIT LINE. HERE WHAT WE’RE DOING, WE’RE CREATING DATA. LET ME SEE IF I HAVE RAN THIS BEFORE. YEAH. WE’RE PLOTTING THE DATA. HERE IS WHAT WE GET. THE SLIDES HAD TENSORFLOW CON ASSISTANTS AND THIS IS TENSORFLOW VARIABLES AND BASICALLY CONSTANTS ARE CONSTANTS AND VARIABLES ARE ADJUSTED OVERTIME. YOU NEVER NEED TO WRITE CODE THIS LOW LEVEL, THIS IS PRETENDING WE DON’T HAVE ANY BUILT-IN FIT METHODS AND WE WANT TO DO THIS FROM SCRATCH. HERE IS HOW YOU DO IT FROM SCRATCH. CREATING VARIABLES. YOU SLICE IT TO 0 AND YOU INITIATE IT TO A NUMBER AND THIS WORKS TOO. THIS DOESN’T LOOK SCARY TALL. THIS IS THE PREDICT FUNCTION. O THIS THE EQUATION FOR A LINE. Y EQUALS MXB AND THE GOAL IS TO FIND GOOD VALUES FOR THIS. HERE IS THE LOSS FUNCTION. WHAT WE’RE DOING, WE’RE TAKING THE RESULTS THAT WE PREDICTED, MINUS THE RESULTS WE WAND. WE’RE SQUARING IT, TAKING THE AVERAGE. THAT IS SQUARED THERE. THEN AS WE GO THROUGH THE NOTEBOOK, WE SEE THE SQUARED ERROR WHEN WE STARTED. THIS IS THE DISSENT FROM SCRATCH PRETENDING WE DIDN’T HAVE ANYTHING LIKE MODEL.FIT. FOR A NUMBER OF STEPS, WHAT WE’RE DOING, WE’RE TAKING OUR XS, WE’RE GOING THROUGH THE MODEL TO PREDICT THE YS, WE’RE GETTING THE SQUARED ERROR, IT IS THE SINGLE NUMBER, AND THEN WE’RE GETTING THE GRADE I CAN’T WANTS OF MNB WITH RESPECT TO THE LOSS. THIS WILL LITERALLY TELL US IF YOU PRINT THEM OUT, THEY’RE JUST NUMBERS, AND THE GRADE I CAN’T WANTS POINT IN THE DEEPEST ASCENT. IF YOU LOOK AT GRADIENTS, LOSS WILL INCREASE, WE WANT THAT TO DECREASE, WE GO IN THE REVERSE MOTION AND THIS IS THE LOWEST POSSIBLE LEVEL WAY TO WRITE THIS CODE. WE’RE DOING THE DISSENT FROM SCRATCH SO WE DON’T HAVE ANY OPTIMIZER, WHAT WE’RE DOING, WE’RE TAKING A STEP IN THE NEGATIVE AND MULTIPLYING THAT BY THE LEARNING GRADE AND THAT ADJUSTS AS WE GO. IF YOU RUN THIS CODE, YOU WILL SEE THE LOSSES DECREASING. YOU WILL SEE THE FINAL VALUES FOR MNB AND PLOT THE BEST FIT LINE. WHAT I DID FOR YOU, I WROTE THIS IS A LITTLE BIT UGLIER, I WROTE CODE TO PRODUCE THIS DIAGRAM SO YOU COULD SEE EXACTLY WHAT THE GRADIENT DISSENT IS DOING. THAT’S HOW YOU WOULD WRITE THINGS FROM SCRATCH AND IN TENSORFLOW 2. WHAT’S AWESOME, WHEN YOU MOVE TO THINGS LIKE NEURAL NETWORKS, THIS CODE IS BASICALLY COPY AND PASTED. IF YOU COMPARE THIS CUSTOM TRAINING LOOP FOR LINEAR PROGRESSION TO THE CUSTOM TRAINING OR FANCY MODELS IN THE WEBSITE, IT IS ALMOST IDENTICAL, THIS IS THE TAPE AND THE STEPS ARE THE SAME. YOU MAKE THE PREDICTION, YOU GET THE LOSS, YOU MAKE THE GRADIENTS AND GO FROM THERE. THAT’S REALLY NICE. IN TERMS OF DATA SETS, YOU BASICALLY HAVE TWO OPTIONS IN TENSORFLOW 2 IT. THE FIRST OPTION, AT THE TOP, AND THESE ARE ALL OF THE EXISTING DATA SETS THAT YOU FIND AND THEY’RE GREAT AND GOOD TO START WITH, THEY’RE USUALLY REALLY TINY, THEY FIT IN MEMORY, NO PROBLEM. THEN WE HAVE AN ENORMOUS AMOUNT OF RESEARCH DATA SETS, THAT’S AWESOME, THAT’S TENSORFLOW DATA SETS. HERE I SHOW YOU HOW TO DOWNLOAD SOMETHING LIKE CYCLE HERE. WHAT’S IMPORTANT TO BE AWARE OF, I JUST HAVE A COUPLE OF QUICK TIPS, IF YOU’RE DOWNLOADING A DATASET IN TENSORFLOW DATA FORMAT, BY THE WAY, IT WILL GIVE YOU SOMETHING CALLED THE DATASET, IT WON’T BE THIS, BUT IT WILL BE IN TF DATA. TF DATA, IT IS A HIGH PERFORMANCE FORMAT FOR DATA, IT IS SLIGHT LIT TRICKIER TO USE THAN WHAT YOU MAY BE USED TO. IF YOU’RE USING TENSORFLOW DATA SETS, YOU HAVE TO BE VERY, VERY CAREFUL TO AVERAGE BECOME MARK THE INPUT PIPELINE. IF YOU JUST IMPORT A DATASET, YOU TRY TO CALL MODEL.FIT ON THE DATASET, IT MAY BE SLOW. IT IS IMPORTANT TO TAKE TIME AND MAKE SURE THAT THE DATA PIPELINE CAN READ IMAGES OFF DISK AND THINGS LIKE THAT EFFICIENTLY. SOME TIPS THAT I MA BE HELPFUL, THEY RECENTLY ADDED AN IN MEMORY FLAG, IF YOU DON’T WANT TO WRITE INTO THE PIPELINES, YOU CAN PASS IN THE MEMORY AND INSERT THE WHOLE THING INTO RAM MAKING IT EASY. ALSO YOU HAVEY CACHING FUNCTION, IT IS REALLY NICE. HERE IS CODE FOR THE TF DATA. MAYBE WE HAVE AN IMAGE DATASET AND WE HAVE SOME CODE TO PREPROCESS THE IMAGES. LET’S PRETEND THAT THAT PREPROCESSING CODE IS EXPENSIVE AND WE DON’T WANT TO RUN IT EVERY TIME WE NEED TO RUN IT. YOU CAN ADD THIS CACLE LINE AT THE END, IT WILL KEEP THE RESULTS AND MAKE SUBSEQUENT RUNS OF THE PIPELINE HANDIER. THIS IS GOOD TO BE AWARE OF. THE GOAL, IT IS TO NOT GIVE YOU ALL THE DETAILS BUT POINT YOU TO THINGS THAT ARE USEFUL. YOU CAN CACE TO FILES, YOU CAN PASS A FILE NAME, YOU CAN ACTUALLY CACLE TO A FILE ON DISK 2. — CACHE. A THING THAT’S AWESOME IN TENSORFLOW 2 IF YOU’RE AN EXPERT YOU’LL CARE ABOUT, IF NOT YOU’LL CARE ABOUT IT DOWN THE ROAD, DISTRIBUTED TRAINING. I’LL SKIP SOME SLIDES TO MOVE FASTER. WHAT I WANTED TO SAY BRIEFLY, EXTENDED TRAINING, IT IS AWESOME BECAUSE IF YOU’RE DOING SINGLE MACHINE MULTIPLE GPU ASYNCHRONOUS DATA PARALLEL TRAINING OR MULTIMACHINE MULTIGPU ASYNCHRONOUS DATA PARALLEL TRAINING, YOU DON’T NEED TO CHANGE THE CODE OF THE MODEL. IT IS EXACTLY WHAT I CARE ABOUT. IT IS AWESOME. BASICALLY HERE IS SOME MODELS THAT ARE BUILT-IN APPLICATION, IT DOESN’T MATTER, I WANT TO SHOW YOU HOW TO RUN THIS CODE ON ONE MACHINE WITH MULTIPLE GPUS. WE WRAP IT IN A BLOCK. THAT’S IT. MODEL.FIT IS DISTRIBUTE AWARE AND WILL WORK. YOU DON’T NEED TO CHANGE THE MODEL TO RUN ON MULTIPLE GPUS AND THIS PARTICULAR STRATEGY IS CALLED THE MIRRORED STRATEGY. THERE ARE DIFFERENT STRATEGIES TO USE TO DISTRIBUTE THE MODELS. THERE IS ANOTHER STRATEGY, LIKE MIRRORED MULTIWORKER, YOU CAN CHANGE THAT ONE LINE, IF YOU HAVE NETWORK WITH MULTIPLE MACHINES ON IT, AGAIN, THE CODE DOESN’T CHANGE. THAT’S AWESOME. I REALLY, REALLY LIKE THE DESIGN OF THIS. ALL RIGHT. OTHER THINGS THAT ARE AWESOME ABOUT TENSORFLOW 2 THAT I ENCOURAGE YOU TO CHECK OUT, ESPECIALLY IF YOU’RE LEARNING OR HAVE STUDENTS, IT IS GOING BEYOND PYTHON. WE TALKED ABOUT TRAINING MODELS AND I WANT TO SHOW YOU COOL THINGS YOU CAN DO TO DEPLOY THESE. ROUGHLY, THERE IS A BUNCH OF WAYS TO DEPLOY MODELS. THE WAY I WAS USED TO, AS A PYTHON DEVELOPER, I THROWED UP A REST API WITH TENSORFLOW SURVEY, PLASTIC PLASTIC AND IT BEHIND THE API. THERE ARE THINGS I HAVE LEARNED SINCE THEN, IT IS GREAT, AND THAT’S BASICALLY DEPLOYING MODELS IN THE BROWSER WITH TENSORFLOW JS, DEPLOYING THEM ON ANDROID AND IOS WITH TENSORFLOW LIGHT AND RECENTLY RUNNING THEM WITH LIGHT MICRO. I HAVE A COUPLE OF SUGGESTED PROJECTS FOR YOU I WANTED TO POINT YOU TO. THE FIRST, IT IS TINY ML, AND THIS IS A BLOG POST, IT WAS A GUEST ARTICLE IN THE BLOG A FEW WEEKS AGO BY A TEAM AND IT IS AN ARTICLE THAT’S BASICALLY A TUTORIAL. WHAT WE’RE LOOKING AT, ITS A MICRO CONTROLLER, AND IF YOU’RE NEW TO MICRO CONTROLLERS, IT IS A SYSTEM ON A CHIP BUT IT ALSO HAS PINS THAT CAN RUN VOLTAGE TOO OR READ VOLTAGE FROM. FOR INSTANCE, YOU CAN PLUG IN LED LIGHT BULB IN A PIN AND YOU CAN HAVE C CODE WHICH RUNS VOLTAGE TO THE PEN AND TURNS ON THE LIGHT. LIKEWISE, YOU COULD HAVE AN ACCELEROMETER ATTACHED TO A PEN AND INTEREST OF HAVE CODE THAT READS FROM THE ACCELEROMETER AND GIVES YOU TIME SERIES DATA. THAT’S WHAT A MICRO CONTROLLER IS, COMPUTER PLUS THE PINS YOU CAN READ AND WRITE TO. TENSORFLOW LIGHT IS THE CODE TO DEPLOY TENSORFLOW MODELS ON TO PHONES. RECENTLY TENSORFLOW LIGHT MICRO NOW LET’S YOU DEPLOY THEM ELSEWHERE AND THESE ARE SMALLER THAN A STICK OF GUM. THIS IS A REALLY NICE ONE, IT HAS — THIS IS A NANO ABOUT WITH BUILT-IN SENSE SENIORS, I HAVE IT AT HOME, IT IS $30, IT HAS A BUILT-IN COMPEL ONLY CENTER, TEMPERATURE SENSOR, — BUILT-IN ACCELEROMETER, TEMPERATURE SENSOR, ALL OF THAT. USING THIS, SOMEBODY BUILT A MODEL ON RECOGNIZING A PUNCH AND UPPERCUT. AS THEY MOVE THAT, THE LAPTOP RECOGNIZES THE GESTURES. THE WORKFLOW, THIS IS IN THE BLOG POST, IT IS NO THE BAD AT ALL CONSIDERING HOW MUCH POWER YOU’RE GETTING OUT OF IT. WHAT YOU’RE DOING, LET ME SEE IF I CAN SHOW YOU THE STEPS. BASICALLY YOU HAVE TO CAPTURE DATA. I WANTED TO BRING THAT WITH ME, I THOUGHT IT WOULD BE BETTER TO JUST SHOW YOU THIS. YOU CAN PLUG THAT IN THE LAPTOP WITH A USB CABLE. YOU NEED TO COLLECT TRAINING DATA FOR THE MODEL. YOU HOLD THIS IN THE HAND. YOU COLLECT THE DATA FOR THE PUNCHING GESTURE AND SAY IT IS A SERIES AND THE DIAGRAM, I WAS JUST CAPTURING THAT THIS MORNING. AS YOU MOVE THE TOOL AROUND, YOU GET THE TIMED FILE LIKE IF YOU WERE LOOKING THROUGH THE TIME SERIES PORTFOLIO. YOU DO THE SAME THING WITH OTHER GESTURES. YOU GATHER DATA, YOU SAVE THE FILES, YOU UPLOAD THEM, IN CODE LAB YOU WRITE A MODEL TO CLASSIFY THE DATA. THAT’S JUST A REGULAR PYTHON MODEL. YOU DON’T NEED TO KNOW ANYTHING SPECIAL ABOUT TENSORFLOW LIGHT TO DO IT. WHAT I WANT TO SHOW YOU, AND YOU CAN FIND THE COMPLETE BLOG IN THE BLOG POST, THERE IS A SMALL AMOUNT OF CODE YOU NEED TO CONVERT THE MODEL FROM PYTHON DOWN INTO TENSORFLOW LIGHT FORMAT TO RUN ON DEVICE. THIS IS A TINY AMOUNT OF CODE. ONCE YOU HAVE THIS MODEL, THIS IS A REALLY UNUSUAL, WE’RE GOING TO CONVERT IT TO A C ARRAY BECAUSE THERE IS ONE MEGABYTE OF DISK SPACE AND 36KB OF RAM. WE’RE CONVERTING IT TO THE SIMPLEST FORMAT, BUT YOU CONVERT IT TO A C ARRAY AND THEN WHAT YOU DO, WE HAVE AN EXAMPLE AND YOU PASTE THAT IN THE EXAMPLE AND THEN YOU RUN THE TENSORFLOW MODEL ON DEVICE. IT IS AMAZING. I HAVE HAD SO MUCH FUN DOING THIS. THE REASON I WANT TO POINT IT TO YOU, YOU HAVE KIDS, STUDENTS THAT WANT TO PLAY WITH THIS STUFF, IF YOU TEACH THEM JUST TO TRAIN A TIME SERIES FORECASTING MODEL.>>A VALUABLE SKILL, A GOOD TUTORIAL FOR IT, IT IS INTERESTING, BUT VASTLY MORE INTERESTING IF YOU THEN DEPLOY IT ON IT DEVICE. IT GIVES THEM SOMETHING TANGIBLE TO PLAY WITH, SHOW FRIENDS AND IT IS SUPER COOL. ALSO THIS IS A BRAND-NEW AREA. TINY ML REFERRING TO MACHINE LEARNING ON SMALL DEVICES HAS A LOT OF PROMISE. ANOTHER WAY TO DEPLOY THE MODELS, SUPER POWERFUL, JAVASCRIPT, THIS IS A PROJECT SUGGESTION. ONE OF THE FIRST TUTORIALS YOU RUN THROUGH, IT IS SENTENCE ANALYSIS, YOU PREDICT IF IT IS POSITIVE OR NEGATIVE, A VALUABLE SKILL COULD BE SOMEWHAT SOMEWHAT DRY. THE FIRST TIME I HAVE EVER BEEN SUPER EXCITED ABOUT SENTENCE ANALYSIS OTHER THAN MANY YEARS WHEN I LEARNED I COULD MAKE MONEY WITH IT, IF YOU GO TO THIS, YOU CLICK FROM THE SLIDES, YOU WILL FIND THAT YOU CAN FIND IT NOW LIVE IN JAVASCRIPT. FOR EXAMPLE, THIS IS A WEBPAGE, AND DOWN AT THE BOTTOM HERE, THE MOVIE WAS AWESOME. I WROTE THAT SENTENCE. YOU SEE THE HE PREDICT IS POSITIVE. THERE YOU SEE IT IS PREDICTED NEGATIVE. WHAT’S NICE ABOUT JAVASCRIPT IN THE BROWSE FERRY A USER PERSPECTIVE, THERE IS NOTHING TO INSTALL. IF YOU’RE A PYTHON DEVELOPER AND THE GOAL IS TO HAVE A COOL DEMO, INSTEAD OF THROWING UP A REST API, YOU CREATE A WEBPAGE AND SHARE IT WITH FRIENDS. WHAT’S NICE, THIS MODEL WAS WREN IN TENSORFLOW 2 AND WE HAVE A CONVERTER SCRIPT THAT WILL CONVERT IT IN JS FORMAT TO RUN THE BROWSER. FOLLOWING THE EXAMPLES, I’M NOT A JAVASCRIPT DEVELOPER TALL. I CAN GET THROUGH THE EXAMPLES AND IT IS NOT THAT BAD. IT IS POSSIBLE TO DO. IT IS A GOOD OPPORTUNITY IF YOU HAVE FRIENDS THAT ARE GOOD DEVELOPERS, THIS IS A HUGE COLLABORATION OPPORTUNITY TO DEVELOP MODELS AND FRIENDS WILL DEPLOY THEM IN THE BROWSER WITH YOU. ALSO IF YOU HAVEN’T SEEN IT, YOU CAN ALSO WRITE MODELS FROM SCRATCH IN JAVASCRIPT AND I WANTED TO SHOW SOME DEMOS HERE. I’LL ACCIDENTLY UNPLUG THIS 9 LAPTOP ALMOST CERTAINLY. ANOTHER SUPER CONVINCING THING ABOUT WHY YOU MAY WANT TO DO JAVASCRIPT IN THE BROWSER, THIS COULD BE THE END OF THE PRESENTATION. THIS IS A MODEL CALLED POSED NET. IT IS RUNNING ENTIRELY CLIENT SIZE ON THE BROWSER, NOTHING IS GOING TO THE SERVER. IT IS NOT MEANT FOR THIS MANY PEOPLE. YOU CAN SEE IT IS RECOGNIZING WHERE PEOPLE ARE IN THE AUDIENCE. WHAT’S COOL, I KNOW THIS IS OBVIOUS FOR WEB DEVELOPERS. I’M NOT. THIS IS NEW FOR ME. FOR ME TO DO THIS IN PYTHON WOULD BE A NIGHTMARE. I WOULD HAVE TO BE STREAMING DATA FROM THE VIDEO CAMERA, SENDING IT TO A SERVER, CLASSIFYING THAT, SENDING RESULTS BACK, THERE IS NO WAY I WOULD BE ABLE TO DO THAT REALTIME LIKE THAT. FOR PRIVACY REASONS, THAT’S NOT COOL. BECAUSE THIS IS RUNNING CLIENT SIDE IN JAVASCRIPT WE CAN DO THAT. IMMEDIATELY, LIKE YOU CAN SEE ALL OF THE THINGS WE CAN DO ON TOP OF THIS. THERE IS OTHER MODELS LIKE THAT TOO THAT ARE REALLY COMPELLING. FOR PEOPLE LOOKING FOR APPLICATIONS, SO THERE IS SENTIMENT ANALYSIS, BUT A MODEL BUILT ON TOP OF THAT, IN THE BROWSER, IT IS JUST TEXT CLASSIFICATION. THIS IS MULTICLASS TEXT CLASSIFICATION AND THIS IS A TOXICITY DETECTOR THAT’S SENTIMENT ANALYSIS MODEL. YOU’RE GIVEN A SENTENCE AND YOU WANT TO FIGURE OUT IF THIS IS A COMMENT YOU WANT TO POST ON YOUTUBE OR SOMETHING LIKE THIS. YOU BUILD TOOLS LIKE THIS THAT ANALYZE TEXT PRIVATELY, QUICKLY, CLIENT SIDE, YOU CAN IMAGINE FOR EXAMPLE IF YOU HAVE A JOB AS A WIKIPEDIA EDITOR, YOU WANTED TO SEE IF YOU WANT TO PUBLISH IT OR NOT, YOU MAY SPEND TIME LOOKING FOR TOXIC COMMENTS. WITH THIS, YOU CAN IMMEDIATELY HAVE CODE RIGHT ON THE WEBPAGE, HIGHLIGHT THE BAD PARTS OF THE ARTICLE AND I NEE TO MOVE A LITTLE BIT QUICKER. LET ME POINT YOU TO ONE DEMO AND I’LL STOP. IF YOU’RE NEW, PROBABLY A BEST DEMO, THERE IS TWO RIGHT HERE, ONE IS POSE NET, YOU GET THAT AT THE LINK, THERE IS ALSO PACMAN, IF YOU HAVEN’T SEEN IT, YOU CAN CONTROL THAT WITH THE FACE, YOU CAN TRAIN HIM IN THE BROWSER TO DO THAT, IT IS AWESOME, AND THEN FLYING THROUGH THIS, LAST COMMENT, THEN LEARNING MORE, IF YOU’RE WORKING IN COLAB AND USED TO USING HAD THIS, YOU WANT TO IMPORT IT FROM TENSORFLOW, THAT WILL GIVE YOU THE VERSION THAT’S THE SUPER SET. THAT’S ONLY A PROBLEM IN COLAB BECAUSE IT IS IN THERE AS DEFAULT. IF YOU SEE TENSOR NOPE BACK END, YOU ARE IN THE WRONG VERSION. LAST SLIDE, HERE ARE FOUR BOOKS THAT I WOULD RECOMMEND. THE FIST, THE FIRST BOOK, IT IS ABOUT TENSORFLOW 2. THIS IS GIVING YOU LOW LEVEL DETAILS, IT IS GREAT. ONLY BUY THE SECOND EDITION! THE FIRST EDITION TEACHES TENSORFLOW 1 W 1 WHICH YOU DOT NEED FOR TENSORFLOW 2. THE SECOND BOOK, IT DOESN’T MENTION TENSORFLOW 2 AT ALL, IF YOU DO DEEP LEARNING, IT IS A PERFECT PLACE TO S START THE NEXT BOOK, DEEP LEARNING WITH JAVASCRIPT, ITS GREAT. THE FOURTH BOOK, GRAND BRAND-NEW. THANK YOU VERY MUCH. I’LL STOP THERE. I’LL BE AROUND AFTER FOR QUESTIONS.>> WE’LL BE BACK IN ABOUT 10 MINUTES, SEE YOU SOON.>>Moderator: I’M ASHLEY. I’M YOUR HOST TODAY. A PLEASURE AND HONOR TO WELCOME OUR NEXT SPEAKERS PAIGE BAILEY AND BRENNAN SAETA WHO WILL TALK TO US ABOUT SWIFT FOR TENSORFLOW, A MACHINE LEARNING PLATFORM THAT LEVERAGES INNOVATION. PLEASE HELP ME IN WELCOMING PAIGE AND BRENNAN. [APPLAUSE]>>Paige Bailey: GOOD MORNING, EVERYONE. THANK YOU FOR COMING TO JOIN US DURING LUNCH TIME. WE VERY MUCH APPRECIATE YOU TAKING THE TIME TO COME TO TENSORFLOW WORLD AND TO LEARN MORE ABOUT SWIFT. I’M PAIGE, THE PRODUCT MANAGER.>>>>Brennan Saeta: I’M BRENNAN.>>Paige Bailey: WE WANT TO GET A FEEL FOR THE ROOM. HOW MANY OF YOU HAVE USED SWIFT BEFORE? EXCELLENT. HOW MANY OF YOU ATTENDED PARIS AND TIM AND THEIR SESSION, THEIR WORKSHOP ON EARLIER THIS WEEK? A FEW HANDS. COOL. SO THIS IS A GREAT PLACE TO GET STARTED IF YOU’VE NEVER USED SWIFT BEFORE. AND IT IS ALSO A WONDERFUL PLACE TO LEARN HOW SWIFT IS GETTING MORE AND MORE FUNCTIONALITY ADDED SPECIFICALLY FROM MACHINE LEARNING AND DEEP LEARNING. IT’S AN EXCITING TIME. DO YOU WANT TO GET STARTED?>>Brennan Saeta: LET’S DO IT. SO TODAY WE WANT TO BRING YOU A LITTLE BIT DIFFERENT KIND OF A — HOW MANY OF YOU KNOW SWIFT? AT LEAST TO SOME DEGREE. WE WANTED THE DIG ONE LEVEL DEEPER AND REALLY EXPLORE SWIFT TENSORFLOW FROM ITS FOUNDATION. SO BEFORE WE GET INTO THE MEAT OF THE TALK I WANTED TO TAKE A MOMENT TO ACKNOWLEDGE SOME VERY IMPORTANT FOLKS. NUMBER ONE IS THIS WORK WOULD NOT BE POSSIBLE WITHOUT THE REST OF THE SWIFT TENSORFLOW TEAM THAT DOES A HUGE AMOUNT OF WORK TO MAKE THIS PLATFORM AWESOME. A HUGE THANK YOU TO THEM. BUT ALSO NOT JUST THE CORE TEAM ITSELF BUT A NUMBER OF OTHER TEAMS WE COLLABORATE WITH ON GOOGLE FROM THE JACK’S TEAM AND TENSORFLOW AND BEYOND. WE REALLY ARE COLLABORATING TOGETHER AND BUILDING ON EACH OTHER’S WORK TO MAKE IT AWESOME. THANK YOU TO GOOGLE AND OUR AWESOME OPEN SOURCE CONTRIBUTORS BEYOND. WE HAVE A LOT OF CONTRIBUTORS THAT ARE REALLY MAKING A HUGE DIFFERENCE TO THE PROJECT. FINALLY, FOR THOSE OF YOU WHO ARE FAMILIAR WITH — SOME OF THE SLIDES WILL BE FAMILIAR, JEREMY AND OTHERS PRESENTED EARLIER THIS YEAR AND I ENCOURAGE YOU ALL TO CHECK THAT OUT. TO BEGIN WITH I WANT TO START WITH A QUESTION. WHY SWIFT FOR TENSORFLOW?>>Paige Bailey: I COULD GIVE MY ANSWER.>>Brennan Saeta: LET’S HEAR IT.>>Paige Bailey: I’VE BEEN USING PYTHON FOR A LONG TIME. I REALLY LOVE KIND OF THE FLEXIBILITY AND AWESOME USERS EXPERIENCE YOU GET USING THE LANGUAGE. IT FEELS VERY UNDERSTANDABLE TO READ THE CODE ON THE PAGE AND READS LIKE A PLAIN SENTENCE AND A VIBRANT ECOSYSTEM OF DATA SCIENCE TOOLING. SOMETHING I WAS ALWAYS REALLY FRUSTRATED ABOUT WHEN I WAS TRYING TO DEPLOY PYTHON MODELS INTO PRODUCTION IS OFTEN YOU HAVE TO DO A LOT OF REWRITES. PART OF THE — PART OF PYTHON’S BENEFITS DON’T NECESSARILY TRANSLATE SO WELL TO DEPLOYMENT SCENARIOS. SO SWIFT TO ME AT LEAST KIND OF A HAPPY WAY TO GET A LOT OF THE GREAT BENEFITS OF PYTHON BUT ALSO THE BENEFITS OF A TYPE LANGUAGE.>>Brennan Saeta: THAT’S RIGHT. SO WHAT WE’RE GOING FOR WITH SWIFT FOR TENSORFLOW THAT GETS YOU FROM RESEARCH INTO PRODUCTION AND MOST IMPORTANTLY THAT EVERYTHING IS CUSTOMIZABLE BECAUSE ALTHOUGH THERE IS A LOT OF FLEXIBILITY YOU NEED TO DO RESEARCH, YOU NEED JUST THE SAME AMOUNT OF FLEXIBILITY AND CAPABILITY OFTEN SOMETIMES SLIGHTLY DIFFERENT TO REALLY MAKE A HIGH-QUALITY PRODUCTION APPLICATION. THAT’S WHAT WE’RE REALLY EXITED ABOUT WITH SWIFT FOR DENSE TENSORFLOW AND WITH THE PLATFORM HOW WE CAN DO INTERESTING THINGS TO OPTIMIZE PRODUCTIVITY. LET’S TAKE A MOMENT TO ASK THE QUESTION WHAT IS SWIFT?>>Paige Bailey: AND SWIFT IS THE PROGRAMMING LANGUAGE THAT WAS CREATED BY CHRIS LATNER PRETTY RECENTLY WHEN HE WAS SPENDING TIME AT APPLE. SINCE THEN CHRIS HAS COME TO JOIN US AND WORKED AT GOOGLE. YOU WILL BE HEARING A LOT ONE OF CHRIS’S OTHER PROJECTS LATER THIS WEEK, MLAR IS IN THE KEYNOTE TOMORROW. SWIFT IS A PROGRAMMING LANGUAGE THAT’S FOCUSED ON THAT EXTENSIBILITY COMPONENT AS WELL AS READABILITY, USABILITY AND PRODUCTION READINESS.>>Brennan Saeta: LET’S REALLY DIG IN AND LET’S REALLY GO FROM THE FOUNDATIONS. I THINK AN IMPORTANT QUESTION TO UNDERSTAND WHAT IS SWIFT AT A DEEPER LEVEL IS TO UNDERSTAND WHAT IS A COMPILER? AND SO AS YOU PROGRAM COMPUTERS, RIGHT, THERE ARE SORT OF TWO FUNDAMENTAL THINGS THAT ARE SORT OF IMMOVABLE. LIKE TO THINK ABOUT THE WORLD IN THEIR OWN PARTICULAR WAYS THAT ARE SORT OF SOMETIMES INCOMPATIBLE. YOU HAVE HUMANS ON ONE HAND THAT WANT TO THINK ABOUT THE WORLD IN A PARTICULAR WAY. AND YOU’VE GOT HARDWARE, WHICH ALSO THINKS ABOUT THE WORLD IN A PARTICULAR WAY. A PROGRAMMING LANGUAGE IS BASICALLY AN INTERMEDIATE POINT IN BETWEEN THESE TWO IMMOVABLE FUNDAMENTAL TRUTHS OF TECHNOLOGY. PROGRAMMING LANGUAGES ARE DESIGNED TO BE EXECUTED ON HARDWARE AND FOR HUMANS TO EXPRESS IDEAS. THERE ARE DIFFERENT POINTS IN THE SPACE YOU CAN PICK. WE THINK THIS IS AN INTERESTING POINT AND WE’LL TALK ABOUT WHY IN A COUPLE OF MINUTES.>>Paige Bailey: ONE OF THE BENEFITS OF HAVING A TYPED LANGUAGE IS THAT A LOT OF THE RESPONSIBILITY THAT MIGHT BE PUSHED ON YOU IF YOU ARE USING A LANGUAGE LIKE PYTHON IS SUDDENLY ABSTRACTED WAY. I DON’T HAVE TO CARE SO MUCH ABOUT BAKING PERFORMANCE OPTIMIZATIONS INTO MY CODE IMMEDIATELY. THE COMPILER CAN JUST DO IT FOR ME. AND ANOTHER GREAT THING IS IF YOU LOOK AT THE SCREEN RIGHT NOW YOU CAN SEE AN EXAMPLE OF A FEW COMMANDS IN SWIFT. IT LOOKS VERY, VERY SIMILAR TO PYTHON. THERE ARE SOME THINGS SPRINKLED IN AND A FEW INDICATIONS OF TYPES. BUT OTHER THAN THAT PRETTY STRAIGHT FORWARD TO UNDERSTAND. IF YOU ARE ONLY EXPERIENCED USING LANGUAGES LIKE C + THIS LOOKS FRIENDLIER.>>Brennan Saeta: IT IS A POWERFUL PROGRAMMING LANGUAGE. SWIFT IS STATICALLY TYPED YOU DON’T ACTUALLY SEE VERY MANY TYPES IN ADVANCE SWIFT CODE. HERE WE ARE DEFINING DICTIONARY NUMBERS FROM STRINGS. THE COMPILER CAN FIGURE THIS OUT FOR YOU. ONE OF THE PRINCIPLES OF SWIFT IS YOU HAVE A VERY HELPFUL COMPILER AND YOU CAN DEFINE HIGH ORDER OPERATIONS, REDUCES, COMMON PATTERN FROM FUNCTIONAL PROGRAMMING ACROSS A SEQUENCE OF ELEMENTS AND YOU CAN THEN CALL THIS X.SUM AND USE KNOW, A NUMBER OF HIGH ORDER FUNCTIONS. WHAT WE’RE TRYING TO GET ACROSS IS SWIFT IS ALMOST AS HIGH LEVEL AS PYTHON. SO THIS IS ACTUALLY A SLIDE I’M BORROWING FROM CHRIS AND JEREMY. AT THE TOP YOU HAVE A SWIFT FOR TENSORFLOW MODEL AND AT THE BOTTOM A PYTHON MODEL. IF YOU LOOK AT THE CODE, IF YOU SQUINT A LITTLE BIT THEY’RE ALMOST EXACTLY THE SAME THING. YOU HAVE A FEW MORE SELF-DOTS ON THE BODY AND CURLY THINGS ON TOP BUT THEY’RE THE SAME THING. SWIFT IS JUST AS HIGH LEVEL, ALMOST, AS PYTHON.>>Paige Bailey: WE ALSO HAVE FEWER LINES OF CODE IN THAT EXAMPLE, IS THAT RIGHT?>>Brennan Saeta: WELL, A LITTLE BIT OF HERE AND THERE. BUT WHAT’S REALLY IMPORTANT ABOUT SWIFT IS THAT IT’S DESIGNED TO BE FAST. IT IS PRACTICALLY IN THE NAME. SO WHO HERE HAS ACTUALLY LOOKED AT ASSEMBLY IN THE LAST SIX MONTHS? FEWER HANDS THIS TIME. YOU DIDN’T EXPECT TO BE LOOKING AT ASSEMBLY TODAY. FOR THOSE OF YOU WHO DON’T KNOW, THE COMPILER EXPLORER IS NEE. YOU CAN PLUG IN SOURCE CODE ON THE LEFT-HAND SIDE AND GIVE YOU ASSEMBLY ON THE RIGHT-HAND SIDE. HERE WE HAVE SWIFT. WE ARE CALLING A HIGHER TO ORDER FUNCTION REDUCE ON A GENERIC TYPE. IT IS AN ARRAY. PASSING IN THE PLUS FUNCTION. REDUCING OVER EVERYTHING. AND IF YOU LOOK AT THE LOOP THAT IS GOING ON HERE, SO YOU HAVE YOUR FUNCTION PRELUDE, A COUPLE CHECKS TO MAKE SURE EVERYTHING IS REASONABLE YOU HAVE THE FIVE INSTRUCTIONS THAT FORM THE MAIN BODY OF YOUR LOOP. YOU DO YOUR AD. CHECK THAT YOU DON’T HAVE OVERFLOWS, INCREMENT ONE TO GET TO THE NEXT VALUE AND COMPARE TO NOT REACH THE END OF THE ARRAY. HAVE YOU HAVEN’T YOU DO THE LOOP AGAIN. THE MOST EFFICIENT INSTRUCTIONS YOU CAN DO. YOU AREN’T WRITING TYPES. THE COMPILER CAN HOPEFULLY OPTIMIZE IT ALL THE WAY DOWN FOR YOU. NOW ACTUALLY, FOR THOSE OF YOU WHO REALLY KNOW ABOUT PERFORMANCE AND ASSEMBLY, A REALLY HELPFUL COMPILER WILL VCTORIZE THIS OPERATION FOR YOU. SWIFT BY DEFAULT IS SAFE. WHEN YOU USE + YOU ARE GETTING SAFE THAT CHECKS FOR OVERFLOWS. IF YOU DO THE AMPERSAND PLUS FUNCTION, AND THE COMPILER WILL HELP REVECTORIZE TO MAKE YOUR OPERATION RUN EXTREMELY QUICKLY. ASSISTIVE DESIGN TO BE SAFE, HIGH LEVEL, BUT OF COURSE ALSO FAST. PROGRAMMING LANGUAGES ARE ALL ABOUT ATOMS AND COMPOSITION. THE ATOMS IN PYTHON ARE C CODE THAT IMPLEMENTS PYTHON OBJECTS AND THE COMPOSITION LETS YOU COMBINE THEM IN INTERESTING WAYS. PYTHON ARAY, PYTHON DICTIONARY. IT IS AS QUICK AS POSSIBLE IN C. AND THEN YOU GET TO BUILD YOUR WHOLE UNIVERSE ON TOP OF IT. C + + TAKES A DIFFERENT TACK. IT IS JUST C PLUS PLUS, THEY’RE IN, FLOAT, C ARAYS AND POINTINGS. A FEW OTHER ATOMS AS WELL. DEVELOP YOUR OWN STRUCTURES. SO THAT’S A C PLUS PLUS. BUT AS YOU THINK ABOUT C PLUS PLUS WHAT’S THE REAL DIFFERENCE BETWEEN STRING AND ARRAY? THEORY BOTH SEQUENCES OF THINGS, RIGHT? AND SO PROGRAMMING LANGUAGE IS ACTUALLY INVOLVED SINCE WHEN PEOPLE’S PLUSES WERE INITIALLY CREATED AND WE CAN DO MORE INTERESTING THINGS TODAY. SO SWIFT BUILDS ON TOP OF A LOT OF THIS PROGRAMMING LANGUAGE, EVOLUTION AND DESIGN. SWIFT IS REALLY — GETTING BACK TO THAT PICTURE YOU’VE GOT HUMANS AND HARDWARE. PROGRAMMING LANGUAGE IS IN THE MIDDLE. YOU HAVE TEXT EDITORS TO HAVE HUMANS INTERFACE WITH THE PROGRAMMING LANGUAGE AND YOU HAVE COMPILERS TO INTERACT WITH THE HARDWARE AND ONCE YOU DEFINE YOUR LANGUAGE IN TERMS OF THE COMPILER. YOU CAN GET RID OF OTHER THINGS AND MAKE A PROGRAMMING LANGUAGE THAT IS MORE POWERFUL. AND SO THE PRIMITIVES IN SWIFT ARE LVM INSTRUCTIONS AND THE CONSTRUCTIONS — IN SWIFT INSTEAD OF FLOAT AND INT BEING DEFINED BY THE LANGUAGE THEY’RE DEFINED IN THE STANDARD LIBRARY SO YOU CAN DEFINE YOUR OWN INT AND FLOAT. NOW THIS MAY SEEM LIKE AN ESOTERIC QUESTION THAT ISN’T TERRIBLY IMPORTANT BUT AS IT TURNS OUT THE DEFINITION OF FLOAT IS SOMETHING THAT’S IN QUESTION THESE DAYS. SO THOSE OF YOU WHO ARE NOT FAMILIAR WITH TPUs, BUT THEY DEFINE THEIR OWN VERSION OF FLOATING POINT FORMAT. IT’S SHORT FOR THE BRAIN FLOATING POINT FORMAT DEVELOPED AS PART OF THE GOOGLE BRAIN RESEARCH EFFORT. WE FOUND ANOTHER FORMAT IS BETTER FOR EFFICIENCY IN HARDWARE AND TRAINING THE NETWORKS. SO SWIFT IS A LANGUAGE THAT IS ESPECIALLY INTERESTING FOR US BECAUSE IT LETS US DEFINE A B FLOAT 16 TYPE THAT WORKS EXACTLY LIKE INFLOWS IN THE REST OF THE LANGUAGE. SO WHAT DOES IT ACTUALLY LOOK LIKE? IS THIS SUPER WEIRD? WHAT IS THIS BLACK MAGIC? SWIFT FROM THE FOUNDATION. LE’S UNDERSTAND WHAT IS THE IMPLEMENTATION OF FLOAT. CHECK IT OUT ON GET HUB BUT HERE IS WHAT IT LOOKS LIKE. STRUCTURE FLOAT AND YOU DEFINE THE VALUE TYPE IN TERMS OF THE LVM THINGS BUILT IN — PLUS EQUALS IS A NORMAL FUNCTION DEFINED IN SWIFT AND IT IS IMPLEMENTED USING THE LVM.>>Paige Bailey: B FLOAT 16 IS VERY IMPORTANT TO US FOR TPUS BUT WE’RE SEEING MORE EXOTIC DATA FORMATS AND SORT OF REQUIREMENTS FOR FLOATS FOR OTHER SPECIALIZED HARDWARE. SO WE ANTICIPATE THIS WILL BECOME EVEN MORE IMPORTANT IN THE FUTURE. AND IT IS A PLACE THAT WE’RE REALLY EXCITED TO EXPLORE MORE WITH SWIFT.>>Brennan Saeta: THAT’S RIGHT. SO EVEN THOUGH INT AND FLOAT ARE ELEMENTS IN THE STANDARD LIBRARY. THEY’RE JUST AS FAST AS WE LOOKED AT THE ASSEMBLY INSTRUCTIONS BEFORE. IT MEANS SWIFT IS A LANGUAGE THAT’S SUPER FLEXIBLE. YOU CAN DEFINE EXTENSIONS ON INT AND THE IS, ODD PROPERTY AS INT. DEFINE AN EXTENSION THAT HAS THUMBS UP OR THUMBS DOWN AND COMPOSE ALL OF THESE EXACTLY AS YOU WOULD EXPECT TO PRINT OUT THESE SILLY TRUE, FALSES AND EMOJIS AND WHAT NOT. BUT SWIFT AGAIN EVEN THOUGH YOU ARE USING THE HIGH LEVEL OPERATIONS IS JUST AS FAST AS C AND THESE NOTEBOOKS ENCOURAGE YOU TO CHECK THESE OUT FROM JEREMY AND CHRIS AND HOW TO BUILD A MAP MOLD JUST AS FAST AS C. THAT’S A LITTLE ABOUT SWIFT. LET’S TALK ABOUT DEEP LEARNING.>>Paige Bailey: YES.>>Brennan Saeta: LET’S HAVE A BRIEF DETOUR BEFORE WE GET THERE.>>Paige Bailey: IT’S ALSO VERY IMPORTANT TO UNDERSTAND THE MATHEMATICS INVOLVED IN IMPLEMENTING A DEEP LEARNING PROJECT, RIGHT? YOU HAVE TO UNDERSTAND A LITTLE BIT ABOUT LINEAR ALGEBRA AND MATH IN ORDER TO EFFECTIVELY ARCHITECTURE EXPERIMENTS. SO FOR THIS NEXT SLIDE WE HAVE A QUIZ AND I PROMISE IT ISN’T TOO SCARY. BUT SOMEBODY IN THE ROOM SHOUT OUT WHAT YOU THINK B WILL PRINT BASED ON THIS SLIDE. I THINK I HEARD A 3. THAT’S CORRECT.>>Brennan Saeta: VERY GOOD.>>Paige Bailey: EXCELLENT.>>Brennan Saeta: THIS IS VALUE SEMANTICS. THIS IS HOW IT WORKS IN PYTHON. EXACTLY WHAT YOU WOULD EXPECT. IF A IS 3, B IS 3. NO MATTER WHAT YOU DO WITH A, 3 IS STILL 3, B IS STILL 3 WE’RE ALL GOOD.>>Paige Bailey: LET’S TAKE A LOOK AT THE NEXT SLIDE. WE HAVE A PYTHON LIST AND THIS CRAZELY ENOUGH WHENEVER YOU ADD 4 WOULD PRINT 3 AND 4 FOR THE VALUES OF B. SO THIS IS NOT VALUES SEMANTICS AND THIS OFFERS A GREAT MANY FRUSTRATIONS FOR DEVELOPERS AS THEY ARE ATTEMPTING TO ARC HITECT.>>Brennan Saeta: STWIFT IS DESIGNED TO BE A VERY FAST PROGRAM LANGUAGE BUT AT THE SAME TIME REALLY PUSH FORWARD VALUE SEMANTICS. AND SO IN SWIFT ARRAYS DO HAVE VALUE SEMANTICS AND BEHAVE AS YOU WOULD EXPECT. A COPY/PASTE FROM A TERMINAL THAT B STAYS THE SAME. A IS ACTUALLY 3 AND 4. BUT IN SWIFT NOT JUST ARRAYS, DICTIONARIES AND OTHER HIGH-LEVEL TYPES THAT YOU BUILD CAN HAVE VALUE SEMANTICS AS WELL. IMPORTANT RAMIFICATIONS FOR HOW TO DO MACHINE LEARNING AND AUTOMATIC DIFFERENTIATION. WHICH LEADS US TO OUR NEXT TOPIC.>>Paige Bailey: AUTOMATIC DIFFERENTIATION IS ONE OF THE PRODUCT-DEFINING FEATURES OF SWIFT. IT IS ONE OF THE THINGS WE’RE MOST EXCITED ABOUT UPSTREAMING TO THE SWIFT PROGRAMMING LANGUAGE. WE SHOWED IN ONE OF OUR SLIDES A FUNCTION PREFACED WITH AT DIFFERENTIAL. THIS IS FUNCTIONALITY THAT ALLOWS YOU TO DIFFERENTIATE NOT JUST SORT OF ASPECTS OF MACHINE LEARNING MODELS BUT LITERALLY ANY FUNCTION IN SWIFT. DO YOU WANT TO SHOW AN EXAMPLE?>>Brennan Saeta: YOU KNOW WHAT? BETTER TO SHOW, NOT TELL. LET’S SEE, HERE WE ARE IN A GOOGLE COLAB, A HOSTED JUPITER NOTEBOOK AND HERE WE DEFINE MY FUNCTION THAT JUST TAKES TWO DOUBLES, DOES SOME MATH OPERATIONS AND RETURNS ON A DOUBLE AND SO TO MAKE THIS FUNCTION DIFFICULTER ENTIONABLE THAT’S HOW YOU ANNOTATE IT. THE COMPILER HAS NOW CONSTRUCTED THE FORWARD PATH BUT ALSO THE BACKWARDS PATH FOR THIS FUNCTION AUTOMATICALLY. TO THEN FIGURE OUT WHAT THE DERIVATIVE OF THAT FUNCTION IS YOU CALL THE GRADIENT FUNCTION. YOU DO THE GRADIENT AT 0.5 OF MY FUNCTION AND THIS GETS YOU THE PARTIAL DERIVATIVE WITH RESPECT TO A. YOU MAY WANT TO TAKE A PARTIAL DERIVATIVE TO A AND B. THAT IS WHAT YOU WOULD EXPECT. YOU HAVE NOW BOTH POINTS. WHEN YOU ARE TRAINING NEURAL NETWORKS YOU ALSO WANT THE VALUE OF THE FUNCTION IN ADDITION TO THE PARTIAL DERIVATIVES AT A PARTICULAR POINT AND SO TO DO THAT INSTEAD OF CALLING GRADIENT YOU CALL VALUE WITH GRADIENT. THIS WILL RETURN A RESULT THAT HAS JUST A — YOU HAVE NAMED TOOPLES. THE VALUE OF THE GRADIENT AND YOU CAN MANIPULATE THEM EXACTLY AS YOU WOULD EXPECT.>>Paige Bailey: IF YOU’RE COMING FROM PRICE — STUCK AROUND FOR THE PRESENTATION EARLIER, THIS WOULD BE SORT OF TAKING THE EQUIVALENT PLACE OF GRADIENT AND TENSORFLOW. INSTEAD OF HAVING TO CREATE A GRADIENT TAPE AND COLLECTING YOUR VARIABLES, YOU CAN DO THE ANTATIONS FOR YOUR FUNCTIONS.>>Brennan Saeta: THIS IS LANGUAGE INTERESTED. THESE AREN’T JUST ON TENSOR OF DOUBLE. THESE ARE REGULAR BUILT IN FLOAT AND DOUBLE AND BEYOND. SO THAT’S HOW YOU WRITE A FUNCTION, CUSTOM FUNCTION AND TAKE THE DERIVATIVE OF IT. WE REALLY WANT TO BUILD THE NEXT PLATFORM THAT ALLOWS FOR MAXIMUM FLEXIBILITY. YOU NEED TO WRITE YOUR OWN TYPE THAT CAN ALSO BE DIFFERENTIATED. LET’S SEE HOW TO DO THAT. HERE TO DEFINE A POINT IN 2D SPACE, X AND Y. WE MARK IT AS DIFFERENTIATED AND WE CAN DEFINE PROPERTIES ON IT. FUNCTION DOT AND HELPER FUNCTION AND YOU CAN COMPUTE THE GRADIENTS OF THIS DOT PRODUCT WITH ITSELF. THROUGH GO. YOU THEN GET THE TANGENT OF THIS POINT AT THAT SPACE. NOW OFTEN IT’S A GOOD IDEA TO MARK YOUR FUNCTIONS AS DIFFERENTIATION BECAUSE THE COMPILER CATCHES ERRORS FOR YOU AND A GREAT FORM OF DOCUMENTATION. WE SEE AN EXAMPLE. PAIGE, THIS SAYS ERROR. WHAT DOES THIS MEAN?>> THE VALUES ARE DISCRETE. YOU CAN’T ADD A LITTLE INCREMENTAL STEP CHANGE.>>Brennan Saeta: DERIVATIVES ARE ABOUT SMALL STEPS. YOU HAVE 0 AND 1. NOTHING IN BETWEEN. THIS FUNCTION IS NOT DIFFICULTER ENTIONABLE. THE COMPILER IS HELPFUL SAYING YOU CAN’T DO IT AND IT’S WHY INSTEAD OF SILENTLY GIVING YOU A WRONG ANSWER. IF WE REMOVE THE ANNOTATION WE CAN GET GOING ON. LET’S SAY WE WANT TO TAKE THINGS TO THE NEXT LEVEL. WHAT IF WE WANT TO DEFINE AN ADDITIONAL PROPERTY SAY LIKE THE MAGNITUDE PROPERTY, WHICH IS DEFINED BY A VECTOR OF THE ORIGIN TO THAT PARTICULAR POINT.>>Paige Bailey: YOU ARE IMPORTING G WEB C. YOU GET ALL THE MATHEMATICS OPERATIONS THAT ARE ALREADY WITHIN THE C — COOL.>>Brennan Saeta: THAT’S RIGHT. A GREAT EXCUSE FOR US TO SHOW THE SWIFT C INTEROP. IN SWIFT YOU CAN IMPORT ANY C HEADER AND USE THE SYMBOL TO DEFINE IT. NO BOILERPLATE, WRAPPERS OR NOTHING. GO STRAIGHT FOR IT. WE’LL IMPORT THIS. THE SQUARE ROOT FUNCTION DEFINED BY C STANDARD LIBRARY. SO WE CAN THEN USE THAT TO DEFINE POINT. THE MAGNITUDE EXTENSION ON POINT THE SQUARE ROOT OF X SQUARED PLUS PI SQUARED. WHAT IS THIS SAYING?>> READ THROUGH, CANNOT DIFFERENTIATE FUNCTIONS THAT HAVE NOT BEEN MARKED DIFFICULTER ENTIONABLE. YOU HAVE TO CHANGE THIS IN ANOTHER FILE.>>Brennan Saeta: SQUARE ROOT IS DEFINED BY THE C COMPILER AND IT HASN’T BEEN TAUGHT HOW TO TAKE DERIVATIVES. A GREAT EXCUSE TO SHOW YOU HOW TO WRITE YOUR OWN CUSTOM DERIVATIVES FOR ARBITRARY FNCTIONS. IF YOU RECALL THE DERIVATIVE OF THE SQUARE ROOT OF X OR WE CAN DEFINE MY SQUARE ROOT UNDERNEATH THE HOOD CALL C. AND WE CAN DEFINE THE DERIVATIVE WITH THIS FUNCTION CLOSURE HERE. SO WE RUN THAT AND THAT COMPILED SUCCESSFULLY. CHANGE SQUARE ROOT TO MY SQUARE ROOT. THAT RUNS AND WE’RE OFF TO THE RACES. WE HAVE NOW DEFINED A MAGNITUDE FUNCTION ON POINT. WE CAN THEN USE THIS FUNCTION INSIDE OTHER DIFFERENTIATED FUNCTIONS LIKE YOU WOULD EXPECT. EVERYTHING FULLY COMPOSES. TAKING POINT AND DOUBLES, ARBITRATE DIFFERENT TYPES AND FIND THE POINT. TAKE THE GRADIENT AND GET THE TANGENT VECTOR DERIVATIVE.>>Paige Bailey: I LOVE THAT YOU’RE ABLE TO USE ALL OF THE GREAT CODE THAT HAS BEEN WRITTEN TO SUPPORT MATHEMATICS BUT I ALSO REALLY LOVE WITH SWIFT YOU GET C AND C PLUS PLUS. AND AS BRENNAN MENTIONED YOU CAN IMPORT ANY HEADER, USE IT AS IS OR EXTEND IT TO MEET YOUR USE CASE. IF YOU’RE WORKING IN AN ENTERPRISE AND YOU HAVE A MASSIVE EXISTING CODE BASE LIKE GOOGLE, AND THEN IT’S REALLY HELPFUL TO BE ABLE TO NOT HAVE TO REWRITE THE ENTIRE WHEEL. YOU CAN JUST IMPORT THINGS THAT PEOPLE HAVE CREATED OVER TIME, USE IT AND THEN EVERYTHING AS PART OF A SWIFT PROGRAM WOULD COMPILE DOWN TO A.SL FILE. PUT IT ON ANY FILE. IT IS REALLY SORT OF THE MOST MULTI-PLATFORM APPROACH THAT I’VE SEEN.>>Brennan Saeta: THAT’S RIGHT. I THINK THIS IS ENOUGH. THERE IS MORE POWER IN THE TENSORFLOW DIFFERENTIATION SYSTEM AND I ENCOURAGE YOU TO MORE DETAILS. LET’S TALK ABOUT NEURAL NETWORKS.>>Paige Bailey: COOL.>>Brennan Saeta: HERE WE GO. SO TELL US ABOUT WRITING NEURAL NETWORKS IN SWIFT.>>Paige Bailey: WE JUST SAW AN EXAMPLE. PRETTY STRAIGHT FORWARD. YOU ONLY HAVE FIVE LINES OF CODE. IT LOOKS VERY SIMILAR TO KAIROS AND IF YOU OPEN UP A COLAB NOTEBOOK YOU CAN SEE AN EXAMPLE OF IT HERE.>>Brennan Saeta: GREAT. SO WHY DON’T YOU WALK US THROUGH THIS RIGHT HERE.>>Paige Bailey: SO YOU CREATE A MODEL BY ADDING SOME LAYERS. HERE WE SEE A CONVOLUTIONAL LAYER, A POOLING LAYER, AND THEN WE CULL A FUNCTION WITH ACTIVE OR WE PREFACE THE FUNCTION WITH THE DIFFERENTIATION AND IT GIVES US OUR GRADIENT.>>Brennan Saeta: WE DEFINE A SEQUENTIAL MODEL JUST IN SWIFT. NOW LET’S JUST THIS AS WE WOULD TO TRAIN THIS MODEL ON SOME DATA. SO HERE TO DEFINE SOME RANDOM TRAINING DATA BECAUSE FOR CONVENIENCE PURPOSES. WE PUT THE MODEL AND OPTIMIZER FOR IT AND WE WILL RUN THIS FOR TEN TRAINING STEPS. WE DO IT IN A FEW LINES OF CODE. HANDWRITING OUT THE TRAINING LOOP UP HERE. WE GET THE VALUE GRADIENT THAT GIVES YOU LOFT AND GRADIENT AND PRINT IT OUT AND USE THE OPTIMIZER TO UPDATE THE MODEL ALONG THE GRADIENT. JUST LIKE THAT WE’RE TRAINING A MODEL AND YOU CAN SEE THE LOSS VALUE IS DECREASING.>>Paige Bailey: IF YOU ARE NOT A FAN OF WRITING OUT AS MANY CUSTOM STEPS AS WE’VE EXPRESSED JEREMY DID A GREAT COLLABORATION WITH CHRIS WHERE HE REIMPLEMENTED FAST AI ON TOP OF SWIFT. IF YOU ARE LOOKING FOR HIGHER-LEVEL CONSTRUCTS YOU JUST WANT THINGS THAT ARE OUT OF THE BOX, I HIGHLY RECOMMEND CHECKING OUT THE NOTEBOOKS AS WELL.>>Brennan Saeta: THAT’S RIGHT. BUT ONE OF THE COMMON WORKFLOWS THAT ANYONE WHO IS DOING MACHINE LEARNING IS — THEY HAVE THE FIRST VERSION OF THE MODEL, TRAIN IT AND THEY FIGURE OUT HOW TO MAKE IT BETTER. THAT’S ONE OF THE GREAT PROMISES OF MACHINE LEARNING. YOU CAN KEEP OPTIMIZING AND INCREMENTING. LET’S SEE HOW TO OPTIMIZE THIS MODEL TO MAKE IT A LITTLE BETTER. YOU MAY BE FOLLOWING A LOT OF RESEARCH AND YOU MAY HEAR THAT SKIP CONNECTIONS AND RESIDUAL NETWORKS ARE A GOOD IDEA. SO IN THE NEXT 30 SECONDS WE’LL CONVERT THIS NETWORK TO BE A RESIDUAL NETWORK WITH A SKIP CONNECTION. READY? HERE WE GO. WE NEED TO DEFINE OUR SECOND DENSE LAYER WE’LL USE AS PART OF THE SKIP CONNECTION. WE’LL DO DENSE FLOAT. I CAN’T TYPE. AND HERE YOU CAN SEE THE AUTO COMPLETE. THAT IS HELPING TO FILL IN THE INITIALIZER PARAMETERS. NOW WE NEED TO ADD IN A SKIP CONNECTION. 10 IS EQUAL TO INPUT AND NEED TO CAPTURE THE VALUE AFTER THE FLATTENED STEP. IT’S EQUAL TO DENSE OF TEMP. NOW WE DO OUR SKIP CONNECTION. DENSE OF TEMP PLUS TEMP 2. NOW WE’VE ADDED AN ADDITIONAL LAYER AND A SKIP CONNECTION TO OUR MODEL RIGHT HERE IN A JUPITER NOTEBOOK. WE NOW RECOMPILE THAT TO MAKE SURE IT’S WORKING WELL. IT LOOKS LIKE IT IS. NOW IT’S A DIFFERENT MODEL AND WE’LL TRAIN IT FOR 10 STEPS AND LOOKS LIKE WE ACTUALLY GOT SLIGHTLY LOWER OR SIMILAR LOCK VALUES. THIS IS GOOD. BUT WE CAN DO BETTER. THIS IS SORT OF AN EXAMPLE OF WORKFLOWS HOW EASY IT IS TO CUSTOMIZE MODELS USING SWIFT FOR TENSORFLOW. FOR THOSE OF YOU WHO ARE JUST APPLYING MODELS TO STANDARD DATASETS, ARCHITECTURE IS THE BREAD AND BUTTER WORKFLOW. YOU NEED TO BE MORE ADVANCED. SO SWIFT FOR TENSORFLOW THE PLATFORM IS INFINITE CUSTOMIZEABILITY. YOU HAVE YOUR OWN CUSTOM LAYER IN A JUPITER NOTEBOOK. SO LET’S SAY IF WE’RE A RESEARCHER AND WANTED TO DEFINE A CUSTOM LAYER THAT WE ADD TWO BIAS TERMS TO THE LAYER INSTEAD OF ONE. IN 10 SECONDS WE’LL WRITE THIS. READY? HERE WE GO. BOOM. ALL RIGHT. GREAT. IT WORKED THIS TIME. ALL RIGHT. SO HERE WE DEFINE OUR WEIGHTS, OUR WEIGHT TENSOR AND FIRST BIAS AND SECOND BIAS TERM. WE HAVE THE CONSTRUCTOR AND FINALLY THE ACTUAL FORWARD FUNCTIONER OR FUNCTION WHERE YOU TAKE THE — THERE WE GO. THIS SHOULD COMPILE. AND ONCE IT DOES THERE WE GO. WE CAN THEN USE IT IN OUR NEW MODEL. INSTEAD OF A DENSE LAYER WE USE A DOUBLE DENSE LAYER AND CAN COMPILE THIS, AND OOPS, THIS DOESN’T QUITE WORK. AGAIN HERE YOU SEE THE SWIFT COMPILER BEING EXTREMELY HELPFUL. YOU HAVE A QUICK TYPO HERE. IT SHOULD BE LABELS. IT TELLS YOU WHAT YOU SHOULD BE DOING AND YOU ARE OFF TO THE RACES. SO WE FIX THAT UP AND CAN RUN THE TRAINING STEPS AND — OKAY, A LITTLE BUG IN OUR IMPLEMENTATION. EXERCISE FOR THE AUDIENCE TO FIGURE OUT WHERE THE BUG WAS. WE’LL LET YOU THINK ABOUT THAT ON YOUR OWN TIME. ANYWAY, THAT’S A BRIEF TOUR HOW TO USE SWIFT FOR TENSORFLOW AS A PRACTITIONER AND APPLYING AND FITTING AGAINST DATASETS AND MODELS. EXCELLENT. WE’LL HEAD BACK TO THE SLIDES. ACTUALLY, I FORGOT. ONE OTHER THING WE SHOWED HOW TO WRITE A CUSTOM MODEL AND HOW TO WRITE A CUSTOM LAYER. IF YOU ARE DOING RESEARCH INTO DIFFERENT OPTIMIZERS YOU WANT TO CUSTOMIZE THAT. OPTIMIZERS IN SWIFT ARE PURE SWIFT CORD. WE HAVE THE SALIENT BIT OF THE OPTIMIZER THAT WE WERE USING JUST NOW. SO IT’S JUST 10 LINES OF CODE TO IMPLEMENT MOMENTUM, AND OTHER ADVANCED FEATURES FOR US TO CAST A GRADIENT DESCENT OPTIMIZER. WE’RE THINKING ABOUT WHAT THE RIGHT WAYS TO FACTOR MACHINE LEARNING TO MAKE IT EASY TO USE, POWERFUL AND FLEXIBILITY AND EXCITED BY WHERE WE ARE TODAY BUT WE’RE ALWAYS LOOKING OF COURSE TO IMPROVE. IF YOU’RE INTERESTED IN PUSHING THE LIMITS, THE STATE-OF-THE-ART THERE ARE A BUNCH OF OTHER THINGS YOU WOULD LIKE TO HAVE YOU TRY AND GIVE US FEEDBACK AND IMPROVE OUR API.>>Paige Bailey: SO WE MENTIONED BEFORE THAT ONE OF THE BIGGEST SORT OF WINS FOR PYTHON IS THAT AN ENTIRE ECOSYSTEM OF DATA SCIENCE PRODUCTS HAVE BEEN BUILT AROUND THIS. WITH SWIFT YOU DON’T HAVE TO GIVE ANY OF THAT UP. BRENNAN HAS A DEMO COMING UP SHOWING THAT YOU CAN USE TOOLING LIKE MAP OR OTHER THINGS OR ANY OF YOUR FAVORITE PYTHON PACKAGES DIRECTLY FROM SWIFT THE WAY YOU WOULD RUN IT IN A PYTHON JUPITER NETWORK.>>Brennan Saeta: THAT’S CORRECT. LET’S TAKE A LOOK. MY SECRET NOTES. THERE WE GO. HERE IS A JUPITER NOTEBOOK WHERE WE IMPORT TENSORFLOW AND IMPORT PYTHON. ONCE WE DO THAT WE CAN USE THE PYTHON OBJECT TO IMPORT ARBITRARY PYTHON LIBRARIES WITH NO WRAPPERS AT ALL. HERE WE’LL IMPORT THE PI PLOT FROM THEM AND NUMB PI. THIS IS SWIFT CODE THAT LOOKS EXACTLY LIKE PYTHON. WE’LL PLOT X AND X. AND JUST TO SHOW YOU THIS REALLY DOES WORK, WE’LL RUN IT AND VOILA, YOU ARE USING THIS RIGHT FROM SWIFT. IN SHORT, YOU DON’T HAVE TO LOSE ALL THE GREAT PYTHON ECOSYSTEM WHILE TAKING ADVANTAGE OF ALL THE CAPABILITIES THAT SWIFT OFFERS. NOW, SURE WHAT ABOUT A MORE WHACKIER LIBRARY? THESE ARE KIND OF WHACKY BUT ONE OF THE REALLY IMPORTANT PARTS OF THE PYTHON DATA SCIENCE ECOSYSTEM ARE ALL THE MACHINE LEARNING LIBRARIES. ONE OF MY FAVORITES IS OPEN I GYM. A PYTHON LIBRARY THAT HAS REINFORCEMENT LEARNING HE I LIBRARIES. IT’S A PYTHON LIBRARY. AS IT TURNS OUT WITH THE SWIFT FOR TENSORFLOW YOU CAN JUST USE IT. HERE WE CAN USE PYTHON TO IMPORT AND DEFINE HYPER PARAMETERS. THE NEURAL NETWORK USING SWIFT FOR TENSORFLOW. WE HAVE TWO DENSE LAYERS. WE HAVE SOME CODE TO INTERACT WITH THE PYTHON ENVIRONMENT INCLUDING DEFINING THE NEXT BATCH. BUT HERE WE USE THAT GYM LIBRARY TO MAKE THE ENVIRONMENT AND GET THE NEURAL NETWORK AND OPTIMIZER AND THEN WE’LL TRAIN IT FOR AS MANY STEPS AS WE NEED. AGAIN, WE SEE THAT THE SWIFT COMPILER IS REALLY HELPFUL TELLING US ONLINE 24 INSTEAD OF ONE HOT LABELS WE PROBABLY NEED PROBABILITIES. IF I TYPE THAT RIGHT WE SHOULD NOW SEE THIS TRAINING. WE’RE USING A NEURAL NETWORK DEFINED IN SWIFT FOR TENSORFLOW DEFINED IN THE PYTHON GYM LIBRARY RIGHT IN A JUPITER NOTEBOOK SEAMLESSLY. YOU CAN TAKE THE RESULTS THAT YOU’VE JUST — THE INTERMEDIATE VALUESCOMPUTED AND PLOT IT USING RIGHT IN THE NOTEBOOK SO IT ALL TOTALLY WORKS ALL BACK AND FORTH.>>Paige Bailey: AS A MACHINE LEARNING ENGINEER I THINK AN IMPORTANT THING TO NOTE WITH THIS IS YOUR WORKFLOW DOESN’T CHANGE. THE ONLY SORT OF INCREMENTAL STEPS YOU NEED TO THAT IS IMPORTING PYTHON AND ASSIGNING –>>Brennan Saeta: EXCELLENT. THAT’S A NUMBER OF DEMOS AND NOTEBOOKS SO YOU CAN SEE THE STATE OF THE WORLD TODAY. BUT LET’S TALK A LITTLE BIT MORE ABOUT WHAT HAS BEEN GOING ON IN THE FUTURE AND THE DIRECTIONS WE’RE HEADING. WE HAVE C INTEROP AND WORKING ON OTHER THINGS AS WELL. HERE IS AN EXAMPLE OF WHAT WE HAVE. SO ON THE LEFT COLUMN YOU DEFINE C PLUS PLUS TYPE HEADER FILE. ON THE RIGHT-HAND SIDE IF YOU DEFINE IT IN EXAMPLE 8 YOU CAN CALL THESE METHODS DIRECTLY ON IT. WE HAVE A NUMBER OF FEATURES ALREADY SUPPORTED INCLUDING TEMPLATES. BECAUSE AS IT TURNS OUT C PLUS, PLUS IS NOT THE MOST SIMPLE LANGUAGE. WE’RE SEEING IT IS AVAILABLE TO BE USEFUL FOR A NUMBER OF THINGS ALREADY TODAY. ONE OF THE MANTRAS IS INFINITE FLEXIBILITY. WE TALKED ABOUT WRITING CUSTOM NETWORKS, LAYERS, OPTIMIZERS, WHAT ABOUT CUSTOM KERNELS AS WELL. WE’VE BEEN WORKING ON THAT. THIS IS SOME OF THE WORK THAT IS PRELIMINARY WORK ON THE TEAM BUT HERE YOU CAN DEFINE 1D AVERAGE POOLING IN PURE SWIFT. THIS ACTUALLY RUNS ON BOTH CPUs AND GPUs. WE ARE STILL GETTING THE FUNDAMENTALS OF THE DESIGN RIGHT. THIS IS THE EXAMPLE OR DIRECTION WE’RE TRYING TO GO WITH THE SWIFT WITH TENSORFLOW SO YOU’RE NOT LIMITED TO WHAT IS IN THE BOX FOR YOU. THE WHOLE WORLD IS YOUR OYSTER. YOU GET TO PROGRAM AT ANY LEVEL OF THE STACK DOING WHATEVER YOU NEED TO DO. YOU’RE NEVER LIMITED WHEN YOU’RE BUILDING ON THE SWIFT FOR TENSORFLOW PLATFORM. WE HAVE HAD A LOT OF AMAZING THINGS HAPPEN AS PART OF THE COMMUNITY THAT’S BEEN GROWINGING AROUND SWIFT FOR TENSORFLOW. WE HAVE A MODEL GARDEN WITH SUPERVISED LEARNING MODELS BUT ONE OF THE COLLABORATIONS I’VE BEEN VERY EXCITED ABOUT HAS BEEN WITH THE DEEP MIND TEAM PART OF ALPHABET. THEY RECENTLY RELEASED OPEN — A COLLECTION OF REINFORCEMENT ENVIRONMENTS AND ALGORITHMS. A SUBSET ARE IMPORTED TO SWIFT. A GREAT PLACE IF YOU WANT TO CHECK IT OUT IF YOU’RE INTERESTED IN REINFORCEMENT LEARNING. SO IF YOU’VE BEEN INTERESTED HOW DO YOU GET STARTED?>>Paige Bailey: YOU CAN GET STARTED WRITING GOOGLE CO-LABS AND A VARIETY OF TUTORIALS AND I ALSO ENCOURAGE YOU TO LOOK AT JEREMY HOWARD’S SWIFT AI IMPLEMENTATION THAT IS AVAILABLE ON GET HUB.>>Brennan Saeta: I WANT TO TAKE A MOMENT TO HIGHLIGHT COMMUNITY PROJECTS. OPEN — ANTHONY HAS DONE AMAZING WORK WITH SWIFT RL. ANOTHER REINFORCEMENT LEARNING LIBRARY AND SWIFT AT ARI LEARNING ENVIRONMENT AND SWIFT PLOT THAT HAS HAPPENED AS A RESULT OF THE GOOGLE SUMMER OF CODE PROJECTS THAT HAPPENED THIS SUMMER THAT PAIGE HAS ORGANIZED THAT ARE REALLY FANTASTIC. ADAM JOINED THE SWIFT FOR TENSORFLOW TEAM THIS SUMMER AND DID GREAT RESEARCH. EVERYTHING IS OPEN SOURCE. WE WOULD LOVE TO HAVE YOU JOIN OUR COMMUNITY. WE HAVE OPEN DESIGN MEETINGS EVERY FRIDAY AT 9:00 A.M. PACIFIC AND LOVE TO HAVE YOU JOIN IN AND HELP SHAPE THIS PLATFORM TOGETHER WITH US. WE RECENTLY RELEASED 0.5 WITH A NUMBER OF IMPROVEMENTSAND STAY TUNED FOR OUR NEXT RELEASES COMING UP. AND WITH THAT I WOULD LIKE TO THANK YOU ALL FOR LISTENING TO US TODAY. [APPLAUSE] WE’LL BE AROUND HERE AND IN THE HALLWAYS THE REST OF THE DAY. FOR THOSE OF YOU HERE IN PERSON AT THE CONFERENCE WE’LL BE HANGING OUT AT THE ASK TENSORFLOW CAFE IN THE EXPO HALL. COME WITH YOUR QUESTIONS AND FEEDBACK AND LOVE TO HEAR WHAT YOU’RE THINKING AND OPEN DESIGN MEETINGS.>>Moderator: A REMINDER WE HAVE LUNCH. THE LUNCH AND TOPICS TABLE HAPPENING IN THE EXPO HALL AND A BETTER TOGETHER DIVERSITY LUNCH SPONSORED BY GOOGLE HAPPENING IN THE MAGNOLIA HALL. ENJOY YOUR LUNCH. P>>Paige Bailey: STATE-OF-THE-ART OF MACHINE LEARNING AND MAKE TENSORFLOW 2.0 RESTRIVE TO MAKE DEVELOPMENT OF MACHINE LEARNED APPLICATIONS EASIER WITH TIGHT INTERROGATION OF EAGER EXCUSE BY DEFAULT. THE GOAL IS TO MAKE THE EXPERIENCE OF DEVELOPING APPLICATIONS WITH TENSORFLOW 2.0 AS FAMILIAR AS POSSIBLE FOR PYTHON DEVELOPERS. OUR DRIVE TO MAKE MORE SIMPLE APIs DOES NOT COME WITH THE EXPENSE OF GIVING YOU THE FLEXIBILITY TO DEVELOP ADVANCE COMPOSITIONS FOR YOUR NEEDS. WE’VE CREATED A MORE LOW LEVEL API. YOU CAN EXPORT TO PROVIDE INHERITABLE INTERFACES WITH CRUCIAL CONCEPTS WITH VARIABLES AND CHECKPOINTS. THIS ALLOWS YOU TO BUILD ON TOP OF THE INTERNAL TENSORFLOW WITHOUT HAVING TO REBUILD TENSORFLOW. FOR EXAMPLE. CREATING YOUR OWN OPTIMIZE LIKE YOU CAN SEE HERE. WE HAVE STANDARDIZED ON THE SAVE MODEL FILE FORMAT TO RUN MODELS IN A VARIETY OF RUN TIMES INCLUDING THE CLOUD, WEB, BROWSER, NO JS, MOBILE AND IMBEDDED SYSTEMS. THIS ALLOWS YOU TO NOT JUST RUN YOUR MODELS WITH TENSORFLOW, BUT DEPLOY THEM TO WEB AND CLOUD WITH TENSORFLOW EXTENDED TO USE THEM ON MOBILE SYSTEMS AND TRAIN AND RUN THEM IN THE BROWERS ON NO JS WITH TENSORFLOW JS. WITH THE DISTRIBUTION STRATEGY API YOU CAN DISTRIBUTE TRAINING WITH MINIMUM CODE CHANGES YIELDING GREAT OUT OF THE BOX PERFORMANCE. DISTRIBUTED TRAINING AS WELL AS WITH CUSTOM TRAINING LOOPS. MULTI-GPU SUPPORT IS AVAILABLE AND TENSORFLOW 2.0 ALSO SPORTS TENSORFLOW RT FOR FAST. CHECK OUT THE GUIDE FOR MORE DETAILS. ANOTHER FEEDBACK ITEM WE HEARD WAS MAKING ACCESS TO DATA EASY WOULD BE A MASSIVE BENEFIT. MUCH OF THE CODE THAT DEVELOPERS WRITE IS IN MANAGING AND PREPARING YOUR DATA SO WE HAVE EXPANDED TENSORFLOW DATASETS GIVING A STANDARD INTERFACE TO A VARIETY OF DIVERSE DATASETS INCLUDING TEXT, VIDEO AND MUCH MORE. WHILE THE TRADITIONAL SESSION-BASED PROGRAMMING MODEL IS STILL MAINTAINED WE RECOMMEND USING REGULAR PYTHON WITH EAGER EXECUTION. THE TF FUNCTION DECORATOR CAN BE USED TO CONVERT YOUR CODE INTO GRAPHS WHICH CAN THEN BE EXECUTED REMOTELY, SERIALIZED AND OPTIMIZED FOR PERFORMANCE COMPLY MENLTED BY AUTO GRAPH BUILT INTO TF FUNCTION TO CREATE CONTROL FLOW INTO TENSORFLOW CONTROL FLOW. OF COURSE, IF YOU’VE USED TENSORFLOW 1.X AND USING FOR A MIGRATION GUIDE WE HAVE AN AUTOMATIC CONVERSION SCRIPT TO HELP YOU GET STARTED. IF YOU WANT TO BUILD APPLICATIONS USING 2.0, TRY OUT OUR ONLINE COURSES WE’VE CREATED TOGETHER WITH THE FOLKS FROM DEEP LEARNING.AI. FOR MORE ABOUT TENSORFLOW 2.0 INCLUDING HOW TO DOWNLOAD AND GET STARTED CHECK OUT THE OFFICIAL TENSORFLOW SITE AT TENSORFLOW.ORG.>> I HAVE TO LOOK AT THE NEWS TO FIND PROBLEMS AROUND THE WORLD. I HEARD ABOUT THE ZIKA VIER USE OUTBREAK AND I WANTED TO USE GENE TECH NEXT AND MALAYSIA AIRLINES GOING MISSING. I CREATED A BLACK BOX FINDER. BUT THEN I FOUND OUT ABOUT FLINT WHICH WAS ONE OF THE BIGGEST PROBLEMS I HAD EVER SEEN. I’M A SCIENTIST AND INVENTOR. IN 2014 FLINT SWITCHED THEIR WATER SOURCE FROM LIKE HERRON WHICH CAUSED LED CONTAMINATION. THE PART THAT HIT ME THE MOST IS THERE ARE ALL THESE CHILDREN WHO DON’T DESERVE UNCLEAN WATER TO DRINK. I WOUND WANT THAT TO HAPPEN TO ME OR MY FAMILY SO WHY SHOULD IT HAPPEN TO THEM? I SAW MY PARENTS TESTING FOR LEAD AND OTHER CONTAMINANTS IN OUR DRINKING WATER. IT WASN’T RELIABLE. THE CURRENT TEST FOR LED CAN TAKE UP TO TWO WEEKS. QUITE EXPENSIVE AND VERY LABOR-INTENSIVE TO GET A RESULT.>> I WANTED TO CREATE A LEAD AND WATER DETECTION TOOL. I CAME ACROSS NEW TECHNOLOGY USING CARBON ENERGY SENSORS AND WANTED TO EXPAND THE IDEA TO DETECT LED IN DRINKING WATER. I WASN’T SURE IF THIS IDEA WOULD EVEN WORK. I NEEDED GUIDANCE AND I NEEDED A LAB.>> WHEN I MET HER I THOUGHT SHE CONDUCTS HERSELF LIKE AN ADULT. SHE IS COMBINING THE SKILLS OF AN ENGINEER, OF A SCIENTIST AND DESIGNER. WE PARTNERED WITH HER. WE HAVE A COMMON GOAL TO PROVIDE SAFE DRINKING WATER.>> WHEN THEY OFFERED ME THE LAB SPACE I COME HOME AND START SCREAMING MY HEAD OFF. OH OR SOMETHING LIKE THAT. I WAS SO EXCITED. I KNEW THAT IF I COULD GET THIS TO WORK I COULD HELP SO MANY RESIDENTS OF FLINT. MY DEVICE IS NAMED AFTER THE GREEK GODDESS OF FRESHWATER. NOW WITH THE 3D TOTALLY WIRED DEVICE. IN ORDER TO TEST THE LEAD IN YOUR WATER YOU ATTACH A DISPOSAL LEAD SENSITIVE CARTRIDGE SPECIALLY TREATED. IF THE WATER HAS LEAD IT STICKS TO THE CHLORIDE IONS AND CAUSES RESISTANCE. THE MORE RESISTANCE, THE MORE LEAD. TO DISPLAY THE RESULTS I WANTED TO CREATE AN APP. SO I USED ANDROID APP MAKER BECAUSE THAT’S WHAT WAS EASIEST FOR ME FOR MY FIRST TIME CREATING AN APP. I CONNECTED THE DEVICE TO THE PHONE USING BLUE TOOTH AND I COULD SEE MY RESULTS.>> SHE IS JUST AMAZING. TAKING ADVANTAGE OF THAT KNOWLEDGE AND THAT PASSION TO BUILD SOMETHING THAT IS FOR THE BENEFIT OF HUMANITY.>> COMING TO FLINT MAKES ME WANT TO WORK HARDER. PEOPLE ARE LINED UP FOR HOURS WAITING TO GET BOTTLED WATER, WHICH WON’T EVEN LAST THEM A WHOLE MONTH. I KNOW MY DEVICE ONLY SOLVES A SMALL PART OF THIS HUGE PROBLEM. BUT IF PEOPLE CAN TEST THEIR WATER THEMSELVES, THAT GIVES THEM THE POWER TO DO SOMETHING ABOUT IT.>> SOCIETY TELLS YOU THAT YOU CAN’T DO CERTAIN THINGS BECAUSE YOU ARE A CHILD OR BECAUSE YOU ARE A FEMALE. SEEING SOMEONE LIKE THIS PERSON SO EAGER, SMART AND PREPARED, SHE IS PUSHING THE ENVELOPE NOT ONLY FOR HERSELF BUT FOR OTHER GENERATIONS TO COME.>> I WANT TO LEARN MORE ABOUT THE WORLD. I WANT TO KNOW ABOUT PEOPLE AND THIS IS MY PATHWAY TO UNDERSTANDING.>> WELCOME ROBBY NEALE TALKING ABOUT TENSE ORS AND TF.TEXT. WE LOOK FORWARD TO HEARING WHAT HE HAS TO SAY. PLEASE HELP ME WELCOME ROBBY. [APPLAUSE]>>HI. CAN YOU HEAR ME? I’M ROBBY. I’M AN ENGINEER AT GOOGLE AND I’LL TALK TO YOU ABOUT HOW TO LEVERAGE TF.TEXT INSIDE OF TENSORFLOW. FOR THOSE UNFAMILIAR WITH LANGUAGE MODELS, THERE WE GO FORWARD, THEY ARE BASICALLY EVERYWHERE. YOU USE THEM IN QUERY UNDERSTANDING. RELATED KEY WORDS SUCH AS ARTICLE SUMMARIES, SPELL CHEEK, AUTO COMPLETE, TEXT TO SPEECH, SPAM FILTERS. YOU CAN’T GET AWAY FROM THEM. IT IS A GOOD TIME TO BE INTO IT NOW. WE ARE GOING THROUGH A RENAISSANCE. LAST YEAR THE PAPER WAS RELEASED THAT USES TRANSFORMERS, I WON’T GO TOO MUCH INTO IT OTHER THAN TRADITIONALLY OBVIOUSLY WHEN YOU ARE WORKING WITH TEXT MODELS DON’T PLAY AS WELL WITH STRINGS. S YOU CAN CONVERT THOSE INTO NUMBERS SO WE’VE USED EMBEDDING DIFFERENT WAYS TO EMBED LIKE GLOVE, ELMO ARE DIFFERENT WAYS TO CREATE DIFFERENT REPRESENTATIONS OF YOUR WORDS. THESE WORK PRETTY WELL. THE ONE PROBLEM IS YOU HAVE WITH SOME WORDS WHEN YOU LOOK THEM UP IN YOUR VOCABULARY, AM I TALKING ABOUT AN ANIMAL OR BASEBALL EQUIPMENTS. WORDS THAT DON’T SOUND THE SAME. THEY ARE SPELLED THE SAME. WHEN YOU ARE TRYING TO REPRESENT THESE AS A VECTOR YOU ARE GETTING — TRYING TO HAVE TWO DIFFERENT MEANINGS TO THE SAME SEQUENCE OF LETTERS. SO WE’VE GOT AROUND IT BY A MODEL THAT USES THE CONTEXT OF THE SENTENCE TO CREATE THE VECTOR FOR THE WORDS. AND SO IT’S DONE REALLY WELL AND THIS IS THE QUESTION AND ANSWER DATASET. BERT WAS LATE 2018. THE SCORES BEFORE THAT WERE IN THE LOW 70s. IT JUMPED UP TO AROUND 82 AND EVER SINCE THEN PEOPLE HAVE BEEN IT RATING ON THIS MODEL THROUGH ROBERTA, EXCEL NET. I PULLED THE SCORES UP FROM LAST WEEK AND YOU CAN SEE THE MOST RECENT MODERN IT’S OUTPERFORMING HUMANS. SO THAT’S PRETTY CRAZY. IT IS EXCITING NOW TO BE INTO THIS. AND SO LET’S JUST JUMP RIGHT IN. WE ARE MAKING PROGRAMMING THESE LANGUAGE MODELS IN TENSORFLOW EASIER. TRADITIONALLY IT’S BEEN VERY DIFFICULT TO DO THIS. LET’S SAY YOU HAVE SOME DATA LIKE HERE IS AN EXAMPLE OF QUERIES. WE WANT TO TRAIN OFF THIS DATA, RIGHT? BEFORE WE CAN DO THAT WE NEED TO DO SOME PRE-PROCESSING WHICH IS BASICALLY TOKENIZATION OUTSIDE OF THAT. WE DIDN’T HAVE THAT AVAILABILITY INSIDE OF TENSORFLOW. ONCE WE DID THE PRE-PROCESSING WE HAD TO SOMEHOW FIT IT INTO A TENSOR SO WE COULD GET THIS PRE-PROCESSED TEXT OUT INTO THE GRAPH AND NORMALLY LIKE PAT OUT OUR TENSORS TO MAKE THEM A UNIFORM SHAPE. WE TRAIN OUR MODEL AND GO TO PUBLISH IT AND PUT IN OUR MODEL SERVER AND WE’RE LIKE OKAY, WE’RE READY TO GO, RIGHT? AND THEN SO WHEN WE GET THE SERVING DATA YOU CAN’T JUST PLUG THAT SERVING DATA RIGHT IN, RIGHT? WE HAD THIS PRE-PROCESSING THAT HAD TO HAPPEN. EITHER YOU ARE RELYING ON THE CLIENT TO TRANSFORM THE DATA AS WELL OR YOU’RE DOING IT YOURSELF, A DIFFERENT LANGUAGE THAN WHAT YOU DID YOUR NORMAL SCRIPTS IN. I’VE SEEN IT WHEN THE PRE-PROCESSING EXACTLY THE SAME. USING THE EXACT SAME — BECAUSE IT’S DIFFERET LIBRARIES, ONE MIGHT CONSIDER A IT’S PROBLEMATIC. OUR CHARTER WAS TO MAKE THIS AS EASY AS POSSIBLE TO SUPPORT TENSORFLOW AND TO DO THIS WE WANT TO DO ALL THE TEXT PROCESSING IN GRAPH. AND THROUGH — WE DO THIS THROUGH A SERIES OF TEXTS AND SEQUENTIAL APIs THAT BETTER REPRESENTS THIS TEXT. IF WE GO BACK TO WHEN IT WAS PAINFUL, WE REALLY WANT TO GET RID OF THIS PRE-PROCESSING STEP, RIGHT? PUT EVERYTHING IN THE GRAPH. SO ALL YOUR PRE-PROCESSING HAPPENS IN GRAPH. WHEN YOU GO TO SERVE THE MODEL YOU AREN’T RELYING ON THE CLIENT TO PERFORM THAT SAME — THOSE SAME STEPS WHEN YOU SERVE THE MODEL AND THEY CALL IT. AND SO REALLY THE MAIN THING THAT WAS MISSING WAS TOKENIZATION. LAST YEAR WE HAD AN ROC WITH TOKENIZER API AND WANTED TO MAKE THIS AS EASY AS POSSIBLE AND STRAIGHT FORWARD. IT IS VERY SIMPLE. AN ABSTRACT TOKENIZER CLASS. IT TAKES INPUT OF STRING AND GIVES YOU BACK YOUR TOKENS. SO IT’S VERY SIMPLE. WE HAVE A COUPLE OF SENTENCES HERE. WE MAKE THEM INTO WORDS. THE ONE THING I LIKE TO POINT OUT WHICH IS NOT COMPLETELY OBVIOUS IMMEDIATELY UNTIL YOU SEE EXAMPLES IS THAT OUR INPUT IS A RANK ONE TENSOR. OUTPUT IS A RANK TWO. THE REASON WHY THIS IS, OUR TOKENS ARE GROUPED BY THE STRING THEY’RE SPLIT FROM. IT’S REALLY EASY FROM THE USER — THE ENGINEER’S PERSPECTIVE TO BE ABLE TO TELL WHICH STRING TOKENS WERE PULLED FROM WHICH STRING IN THE ORIGINAL TENSOR. THE ONE THING YOU CAN’T DO FROM THIS OUTPUT IS TELL WHERE IN THAT ORIGINATING STREAM IT CAME FROM AND FOR THAT WE HAVE LIKE ONE EXTRA TOKENIZER WITHOFF SETS ABSTRACT CLASS THAT WAS TOKENIZER WITH OUTSET. YOU GET AN INPUT TENSOR OF STRINGS AND GIVES YOU YOUR TOKENS AND WHERE THOSE TOKENS START. SO YOU CAN SEE THAT EXAMPLE HERE. WE CALL IT TOKENIZED WITH OFFSET AND WE CAN SEE THE LETTER LIKE I STARTS AT 0 AND MOVES ONE POSITION AND NO IS IN THE SECOND POSITION AND MOVES UP SIX CHARACTERS. SO THROUGH THESE OFFSETS IF YOU WANT TO KNOW WHERE THE TOKENS ARE IN YOUR ORIGINATING STREAM, YOU CAN DO THAT. IN THE — YOU’LL NOTICE THE SHAPES ARE EXACTLY THE SAME AS THE SHAPES OF THE TOKENS. MAPPING TOKEN TO STARTS AND LIMITS IS VERY SIMPLE FROM HERE. SO WE PROVIDE FIVE BASIC TOKENIZERS. ONE OF THE QUESTIONS WHEN WE FIRST DID THE RC WAS WHY DON’T WE JUST HAVE ONE AND LIKE ONE TOKENIZER TO RULE THEM ALL? THE PROBLEM IS EVERY MODEL IS DIFFERENT. YOU HAVE DIFFERENT LIMITATIONS AND THINGS YOU WANT TO GET AROUND AND WE DON’T WANT TO PUSH OUR OPINION ON YOU BECAUSE THEY ARE ALL DIFFERENT. WE JUST WANT TO BUILD THE TOOLS AND ALLOW YOU TO MAKE THE DECISIONS. SO A LOT OF THESE ARE VERY SIMPLE. THE WHITE SPACE PUTS US ON WHITE SPACE. IF YOU KNOW UNICODE CHARACTERS. THEY’RE BROOKD TOGETHER IN UNICODE SCRIPTS. YOU HAVE LATIN CHARACTERS, GREEK, ARABIC, JAPANESE ARE SOME EXAMPLES AND THEY GROUP SPACE, PUNCTUATION AND NUMBERS AS WELL. IT SPLITS IN THE MOST SIMPLE CASE IF YOU ARE WORKING WITH ENGLISH THE MAIN DIFFERENCE BETWEEN WHITE SPACES IT SPLITS OUT THE PUNCTUATION. WORD PIECE, THIS WAS POPULARIZED BY THE BERT MODEL. IT TAKES TEXT THAT YOU’VE ALREADY TOKENIZED AND SPLITS THOSE WORDS INTO EVEN SMALLER SUB WORD UNITS. SO THIS IS ACTUALLY A GREATLY REDUCES THE SIZE OF YOUR VOCABULARY AS YOU TRY TO ENCAPSULATE MORE INFORMATION, YOUR VOCABULARY WILL GROW. BY ACTUALLY BREAKING THE WORDS DOWN INTO SUB WORD UNITS YOU CAN GREATLY GET THAT SMALLER AND ENCAPSULATE MORE MEANING AND LESS DATA. AND IT GENERATES A VOB VOCABULARY AND YOUR OWN GET HUB AND THE ORIGINAL MODEL HAS A VOCABULARY YOU CAN USE. FRONT END PIECE IS VERY POPULAR TOKENIZER SO THIS IS ACTUALLY RELEASED PREVIOUSLY. GET HUBS WHERE PEOPLE HAVE DOWNLOADED IT AND PRETTY POPULAR. TAKES THE CONFIGURATION WHERE YOU SET UP A BUNCH OF PRE-PROCESSING STEPS ALREADY AN FEED IT TO IT AND IT DOES THAT. IT DOES SUBWORD TOKENIZATION WORD AND CHARACTER. YOU CAN USE LIKE I SAID THE WORKPLACE TOKENIZATION AND DO A PRE-TOKENIZATION STEP. SOME NORMALIZATION AND THE WORD PIECE NORMALIZATION. WE NEEDED A WAY TO REPRESENT THESE AND THAT’S WHERE WE CREATED — FOR A BETTER REPRESENTATION OF TEXT. IF WE LOOK AT AN EXAMPLE WE HAVE TWO SENTENCES AND LIKE I SAID NORMALLY YOUR SENTENCES ARE NEVER OF THE SAME LENGTH. SO WHEN YOU TRY TO CREATE A STRATEGY YOU GET A VALUE ERROR. IT NEEDS TO BE A UNIFORM SHAPE. TRADITIONALLY LIKE I SAID PREVIOUSLY WE PADDED OUT THE STRINGS. IN THIS EXAMPLE YOU’RE LIKE OKAY, EXTRA VALUE IS NOT SO BAD. BUT WHEN YOU ARE ACTUALLY RUNNING UP THESE MODELS, YOU DON’T KNOW HOW LONG YOUR SENTENCES ARE GOING TO BE SO YOU HAVE A FAKE SIZE. A LOT OF TIMES THEY SEEM LIKE FIXED SIZE OF 128 WORDS AND YOU JUST HAVE ALL THIS EXTRA INFORMATION THAT YOU DON’T REALLY NEED INSIDE YOUR TENSOR. IF YOU TRY AND MAKE THAT SMALLER WHEN YOU DO HAVE A LONG SENTENCE THOSE SENTENCES ARE TRUNCATED. SO YOU MAY THINK WELL, WE HAVE SPARSE TENSOR. AND THIS IS ALSO NOT QUITE AS GOOD BECAUSE THERE IS A LOT OF WASTE OF DATA THAT YOU ARE HAVING TO SUPPLY FOR SPARSE TENSOR. AS YOU KNOW OR IF YOU DON’T, SPARSE TENSOR IS REALLY INTENSIVE PRO EVERYTHING IS MADE OF TENSORS, IT’S MADE OF THREE. VALUE, SHAPE, WHERE THE VALUES EXIST WITHIN YOUR MATRIX SHAPE. AND SO YOU CAN SEE THERE IS ACTUALLY A PATTERN. RAGGED TENSORS ARE NOT NECESSARILY THAT THEY’RE SPARSE. IT IS DENSE. THEY JUST HAVE A VARYING LENGTH. IT WOULD BE GOOD IF WE COULD SAY THE FIRST ROW HAS ZERO THROUGH FIVE AND ROW TWO ZERO THROUGH TWO AND THOSE MAKE UP OUR SENTENCES. IT IS EASY TO CREATE. YOU JUST HAVE A TF RAGGED CONSONANT TO CREATE IT. IT IS SIMILARLY BUILT LIKE A SPARSE TENSOR. IT IS MADE UP OF VALUES AND ROW SLITS AND MINIMIZES THE WASTE OF INFORMATION SO YOU CAN SEE THAT ALL THE VALUES ARE IN ONE TENSOR AND WE SAY WHERE WE WANT TO SPLIT UP THAT TENSOR TO BUILD UP OUR DIFFERENT ROWS. IT IS EASIER TO KIND OF SEE IT IN THIS FORM WHERE THE GRADE BLOCK ON THE LEFT SIDE IS WHAT THE RAGGED TENSOR IS IN ITS INTERPRETATION. ON THE RIGHT IS HOW IT WOULD LOOK REPRESENTED. AND DOWN BELOW IS HOW YOU WOULD ACTUALLY DO THAT CALL OR BUILD THIS IF YOU USE VALUES INSIDE OF TENSORFLOW. THE ORIGINAL WAY WE HAD THEM. WE HAD PEOPLE COME TO US AND REPRESENTED THESE IN DIFFERENT WAYS. WE ALSO PROVIDE ROW I.D.s WHERE THE I.D. TELLS WHERE THE VALUE IS INSIDE YOUR TENSOR AND ROW LENGTHS THAT SAYS THESE ARE THE LENGTHS OF EACH ROW SO THE FIRST ONE TAKES THE FIRST FOUR VALUES. YOU CAN HAVE EMPTY ROWS OF ZERO, TWO AND SO ON. AND SO WE WANT TO TREAT THESE LIKE ANY NORMAL TENSOR. RAGGED TENSORS JUST LIKE YOU WOULD SEE IT IN NORMAL TENSORS. WE HAVE A RANK OF TWO AND WE ALSO HAVE SHAPE. THE QUESTION MARK WHEN WE FIND OUR SHAPE IS — THAT DENOTES THE RAGGED DIMENSION. IT IS NOT NECESSARILY ALWAYS ON THE END BUT IN THIS CASE IT IS ON THE END. WE CAN USE NORMAL TENSORFLOW FUNCTIONS IN OPS LIKE WE WOULD WITH NORMAL TENSORS. IT GRABS THE SECOND AND THEN THE FIRST ROW. GATHER D WHICH GRABS THE INDEX. CAN GET CAT ON THE DIFFERENT AXIS. WE MADE THIS FOR TEXT PROCESSING. SO YOUR STRING OPS WORK WELL WITH RAGGED TENSORS, WE DECODE THE STRINGS INTO CODE POINTS AND CODE THEM BACK INTO STRINGS. AND CONDITIONALS WORK AS WELL. IN THIS CASE THE WHERE CLAUSE WHERE WE USE RAGGED TENSORS INSIDE. THE ONE CASE HERE IS THAT THE RAGGED TENSORS FOR THE WHERE MUST HAVE THE SAME ROW SLIPS. THE ROWS MUST BE THE SAME LENGTH. AND IT’S EASY TO CONVERT INTO AND OUT OF RAGGED TENSORS, YOU CAN DO FRONT TENSOR OR FROM SPARSE TO CREATE A RAGGED TENSORAND TO MOVE BACK YOU HAVE YOUR RAGGED TENSOR AND CALLED IT RAGGED TENSOR TWO SQUARE AND IT GIVES YOU A LIST IF YOU WANT TO PRINT IT OUT. WE’RE ALSO ADDING SOME — THESE ARE THE LAYERS CURRENTLY AVAILABLE OR COMPATIBLE AND THERE IS A LOT LEFT WE HAVE TO DO ON THIS FRONT. SO WE ARE PUSHING TO GET MORE LAYERS, COMPATIBLE WITH RAGGED TENSORS IF YOU ARE USING THEM AND COME ACROSS THAT’S NOT COMPATIBLE THERE AT THE BOTTOM WE DO PROVIDE A TWO DENSE LAYER THAT WILL CONVERT IT FOR YOU. THE REAL COOL THING I WANT TO POINT OUT IS THE R AND N SUPPORT. WE’VE SEEN ON TESTS THAT WE GET 10% AVERAGE SPEED-UP WITH LARGE BATCHES, LIKE 30% OR MORE. THIS IS VERY EXCITING AND, YOU KNOW, WON’T GO INTO DETAILS BUT IT’S VERY INTUITIVE. IF YOU THINK ABOUT WHEN YOU ARE ARE LOOPING THROUGH THEIR TENSORS, YOU KNOW WHEN YOU’RE AT THE END OF THE RAGGED DIMENSION YOU CAN STOP COMPUTATION. IF YOU’RE USING TENSORS YOU’RE USING MASK VALUES AND YOU HAVE TO KEEP COMPUTING UNTIL THE END OF THE TENSOR WIDTH. YOU HAVE A LOT LESS COMPUTATION AND YOU SAVE A LOT THERE. ALL RIGHT. SO I WANT TO GO OVER A COUPLE EXAMPLES SHOW YOU HOW EASY IT IS TO WORK WITH. YOU FIRST CAN INSTALL TENSOR TEXT. OUR VERSIONS NOW MAP TO TENSORFLOW VERSION. IF YOU USE TENSORFLOW 2.0, USE TENSORFLOW TEXT 2.0. IF YOU USE 1.15 USE TEXT 1.15. THE CUSTOM APPS. YOU CAN IMPORT IT LIKE THIS. WE GENERALLY IMPORT IT. IN THESE EXAMPLES YOU’LL SEE IT WRITTEN OUT TEXT. SO LET’S GO OVER WHAT YOU MIGHT DO. SO NORMALLY YOU WILL GET YOUR INPUT TEXT. HERE WE HAVE A COUPLE OF SENTENCES. WE WANT TO TOKENIZE IT AND PUT THE SENTENCES INTO WORDS AND WE WANT TO MAP THOSE WORDS INTO IDEAS INSIDE OF VOCABULARY THAT WE’LL FEED INTO OUR MODEL. AND SO THE PRE-PROCESS FUNCTION MIGHT LOOK LIKE THIS WHERE WE JUST SUBSTANTIATE THE TOKENIZER, CREATE A RAGGED TENSOR OUT OF THAT INPUT. AND THEN MAP A TABLE LOOK-UP INTO OUR VOCABULARY ALONG THE VALUES OF THAT RAGGED TENSOR. SO IF YOU REMEMBER WHAT THE RAGGED TENSOR LOOKED LIKE UNDERNEATH, WHEN WE HAVE OUR WORDS AND TOKENS WE HAVE THE RAGGED TENSOR ABOVE AND WHERE THE VALUES ARE SET AND THEN THE RULE SPLITS ARE IN A SEPARATE TENSOR. WHEN WE WANT TO MAP THOSE WORDS TO I.D.s WE’RE KEEPING THE SAME SHAPE. WE ONLY WANT THE MAP OVER THE VALUES. AND SO THAT’S WHY THE MAP OVER VALUES IS THERE BECAUSE WE ARE JUST CONVERTING — DOING THE LOOK-UP ON EACH WORD INDIVIDUALLY. AND SO THE RESULTING RAGGED TENSORS THERE AT THE END AND WE CAN SEE WHAT IT ACTUALLY REPRESENTS ABOVE. AND SO THIS IS OUR PRE-PROCESSING. ONCE WE’RE DONE, USING TF DATA. YOU MAP THE PROCESSING OF THE DATASET. YOU CAN CREATE A MODEL WITH KAIROS SIMPLY AND THAT TRAINS THE MODEL. SO YOU CAN USE THAT SAME PRE-PROCESSING FUNCTION IN YOUR SERVING INPUT FUNCTION. SO YOU HAVE THE SAME PRE-PROCESSING THAT’S DONE AT TRAINING TIME AS IT IS IN SERVING TIME WITH THE INFERENCE. AND THIS PRESENTS TRAINING SKEW THAT WE’VE SEEN MULTIPLE TIMES IN THE PAST. I HAVE AT LEAST. SO LET’S GO OVER ANOTHER EXAMPLE. CHARACTER DIAGRAM MODEL HERE. BEFORE I JUMP IN I JUST WANT TO QUICKLY GO OVER END GRAM. IT’S BASICALLY I SAY LIKE A GROUPING OF A FIXED SIZE OVER A SERIES. WE PROVIDE THREE DIFFERENT WAYS TO JOIN THOSE TOGETHER SO THERE IS THE STRING JOIN AND YOU CAN SUM VALUE AND ALSO TAKE AVERAGES. PULL OUT AN EXAMPLE HERE. SO HERE WE’RE DOING A DIAGRAM OF WORDS SO WE HAVE A SENTENCE WE JUST TOKENIZE IT TO SPLIT IT UP INTO WORDS AND THEN WE CALL THE END GRAM FUNCTION IN TENSOR TEXT THAT GROUPS THOSE WORDS TOGETHER WHICH BASICALLY IS JOINING A STRING. SO EVERY TWO WORDS ARE GROUPED TOGETHER AS YOU CAN SEE. THAT’S GENERALLY A BIGRAM. TRIGRAMS IS THREE. YOU CAN SEE HERE WITH SPLIT OUR SENTENCE INTO CHARACTERS AND THEN WE GROUP THEM TOGETHER WITH EVERY THREE CHARACTERS WITH A WIDTH OF THREE. AND THEN IN THIS SITUATION THE DEFAULT OF SEPARATOR IS A SPACE AND SO WE JUST DO THE EMPTY STRING. SO THE OTHER TWO ALSO WORKS WITH NUMBERS. SO IF WE HAVE A SERIES HERE 2, 4, 6, 8, 10 IS OUR TENSOR AND WE WANT TO SUM UP EVERY TWO NUMBERS, 4 PLUS 6 IS 10 AND SO ON AND THEN ALSO AVERAGE, THE MEAN REDUCTION TYPE. WHERE THIS MIGHT — GENERALLY WHEN YOU TALK ABOUT END GRAMS YOU ARE TALKING IN A LANGUAGE CONTEXT. WHERE THIS WOULD BE HELPFUL LET’S SAY IF YOU WERE TAKING TEMPERATURE READINGS EVERY 20 MINUTES AND YOU HAD A SERIES OF TEMPERATURE READINGS EVERY 20 MINUTES. WHEN YOU WANT TO FEED IN YOUR MODEL AS AN AVERAGE OF THOSE TEMPERATURES OVER AN HOUR PERIOD EVERY 20 MINUTES YOU CAN DO A TRIGRAM WITH A REDUCTION OF MEAN. SO IT TAKES THE AVERAGE OF THOSE 20-MINUTE INTERVALS AND SO YOU GET AVERAGE TEMPERATURE OVER THE HOUR AT EVERY 20 MINUTES AND YOU CAN FEED THAT INTO YOUR MODEL. BUT GENERALLY LIKE I SAID, WITH THE BIGRAMS AND TRIGRAMS IT IS OFTEN USED IN NLP AND HOW THAT WORKS IS YOU GENERALLY LIKE SPLIT IT UP EITHER INTO WORDS OR CHARACTERS AND THEN HAVE A VOCABULARY DICTIONARY AND YOU CAN LOOK UP THE GROUPINGS. AND OUR EXAMPLE WE CHEAT A LITTLE BIT. WE GET OUR CODE POINTS FROM OUR INPUT SO WE HAVE THIS INPUT. WE CAN GET CODE POINTS. AS YOU SEE AGAIN, THE RANK IS INCREASED SO WE HAD THE SHAME OF THREE. AND THEN THAT HAD A SHAPE OF THREE WITH A RAGGED DIMENSION AND WE USED MERGED DIMENSIONS TO ACTUALLY COMBINE THOSE TWO DIMENSIONS BECAUSE WE DON’T CARE ABOUT THEM IN THIS CASE. AND SO IT TAKES A SECOND TO LAST AXIS AND COMBINES THEM. THEN WE’RE JUST SUMMING THOSE UP TO CREATE OUR UNIQUE I.D. IN THIS CASE THAT WE’LL FEED INTO THE MODEL. I THINK GENERALLY LIKE I SAID YOU WOULD DO STRING JOINS AND LOOK THOSE UP IN A VOCABULARY BUT FOR THIS CASE MODEL IT WORKS. WE CAST THOSE VALUES. THIS IS OUR PRE-PROCESSING FUNCTION THAT AGAIN WE CREATE A DATASET USING TF EXTRA DATASET. MAP OUR PRE-PROCESSING FUNCTION ON THOSE VALUES AND THE MODEL THAT’S CREATED WE CAN TRAIN USING THIS PRE-PROCESSING FUNCTIONS. ALL RIGHT. FINALLY, I WAS GOING TO GO OVER THE — PROCESSING. A LITTLE BIT MORE CODE IN THIS ONE. I JUST WANT TO SAY WE PROVIDE THE BERT TOKENIZER FOR YOU SO FEEL COMFORTABLE IN KNOWING YOU DON’T REALLY HAVE TO WRITE THIS IF YOU DON’T WANT TO. YOU CAN USE THE BERT TOKENIZER AND IT DOES ALL THIS STUFF FOR YOU. I FEEL THERE ARE A LOT OF GOOD EXAMPLES AND WHAT IT DOES AND IF YOU ARE DOING PRE-PROCESSING, THESE ARE THINGS YOU SHOULD PROBABLY THINK ABOUT AND KNOW ABOUT. SO I WANTED TO GO OVER IT WITH YOU. THIS IS LIKE A SLIM VERSION OF THAT. SO WHAT IT DOES, WHAT BERT DOES IN PRE-PROCESSING, LOWER CASE AND NORMALIZED THE TEXT. AND THEN IT DID SOME BASIC TOKENIZATION. IT SPLIT OUT CHINESE CHARACTERS AND EMOJI BY CHARACTER SPLITTING AND IT DID WORD PIECE ON TOP OF ALL THAT. SO WITH LOWER CASE AND NORMALIZING THIS IS VERY COMMON THAT YOU WOULD DO WHEN YOU ARE OKING UP WORDS IN YOUR VOCABULARY, YOU WANT THE WORDS TO MATCH AND NOT HAVE DUPLICATE WORDS. SO CAPITALIZATION GETS IN THE WAY WITH THAT. WORDS AT THE BEGINNING OF SENTENCE ARE CAPITALIZED. WHEN YOU LOOK IT UP, IT WOULD BE IN YOUR DICTIONARY OR VOCABULARY TWICE. GENERALLY THOUGHT YOU WOULD LOWER CASE THESE. AND NORMALLYIZATION IS A LOT OF UNICODE CHARACTERS WHERE ACCENTS CAN BE REPRESENTED IN DIFFERENT WAYS. AND SO NORMALIZATION NORM -NORMALIZES IT. IT WOULD CONFUSE YOUR MODERN MAKING THE VOCABULARY LARGER. WE PROVIDE CASELOAD WHICH IS JUST AN AGGRESSIVE VERSION OF TWO LOWER. WHAT IT DOES IS IT LOWERS — LOWER CASES CHARACTERS AND ALSO DOES DRK — WORKS WITH NON-LATIN CHARACTERS AND ACCENTED CHARACTERS. IT DOESN’T MESS UP NON-LETTERS. AND DOES NFKC FOLDING AND NORMALIZATION. TALK A LITTLE BIT MORE ABOUT THAT. WE DO THAT IN OUR FIRST STEP. I HAD EXAMPLES OF WHAT THIS WOULD LOOK LIKE. THIS EXAMPLE IS REALLY JUST LOWER CASING OUR I IN IF. AND THEN BERT ACTUALLY NORMALIZED TO NFD. WE’LL NORMALIZE TO NFKC. LETTERS HAVE MANY FORMS WITH A SINGLE NORMALLYIZATION. WHEN YOU WORK WITH FOREIGN CHARACTERS THEY ARE — NOW WE’LL DO SOME BASIC TOKENIZATION AND SPLIT ON THE SCRIPT AND TOKENIZE OUR TEXT. WHAT YOU MIGHT NOTICE HERE IS WHILE OUR SENTENCE ISN’T TOKENIZED THE CHINESE CHARACTER HAVE NOT. THAT’S BECAUSE IT IS A SINGLE SCRIPT THROUGHOUT THAT WHOLE SENTENCE AND THERE ARE NO SPACES OR ANY OTHER METHOD OF DEFINING SEPARATIONS AND WORDS. WE WANT TO DO IS WANT TO SPLIT THAT UP. THIS IS KIND OF LIKE WHERE A LOT OF CODE COMES IN. YOU KNOW, YOU CAN FOLLOW. AND I THINK THE MAIN POINT IS JUST TO KNOW THESE THINGS WE THOUGHT ABOUT AND IF YOU RUN ACROSS IT THERE ARE WAYS TO WORK AROUND THIS. I PREPARED YOU, TRIED TO. IT’S SIMPLE WHEN WE STEP THROUGH IT. YOU’LL SEE. FIRST WE GET THE CODE POINTS — SORRY, YES, WE GET CODE POINTS OF THE CHARACTERS. AND THEN WE JUST KEPT SCRIPT I.D.s OF THOSE CHARACTERS SO YOU CAN SEE THE FIRST SENTENCE IS ALL SCRIPT 17 WHICH IS HAN SCRIPT, CHINESE. OUR LATIN CHARACTERS ARE 25 AND EMOJI AND PUNCTUATION ZERO. WE APPLY MATH.EQUAL ON THE RAGGED TENSOR THAT GIVES YOU — WE WERE JUST TECHING — CHECKING AT THE SCRIPT. WE USE THE SLIGHT NOTATION TO GRAB THE SAME CHARACTER. WE KNOW THEY’RE THE SAME ALREADY. WE WANT TO CHECK FOR EMOJI AND IN TENSOR TEXT WE PROVIDE OUR FUNCTION WORD SHAPE WHICH YOU CAN ASK BASICALLY DIFFERENT QUESTIONS ON WORDS. IT’S BASICALLY LIKE DIFFERENT REGULAR EXPRESSIONS THAT YOU WANT TO ASK. HERE WE’RE ASKING IS THIS — DOES THIS TEXT HAVE ANY EMOJI? OTHER ONES, IS THERE ANY PUNCTUATION? ARE THERE ANY NUMBERS? IS MY STRING ALL NUMBERS? THESE ARE THINGS YOU MIGHT WANT THE FIND OUT ABOUT AND PROVIDE YOU THE METHOD TO DO THIS. SO HERE WE JUST CONDITION TOGETHER TO SAY WHETHER WE SHOULD SPLIT OR NOT. IT WORKS WITH RAGGED AND THEN WE GO AHEAD AND SPLIT EVERYTHING INTO CHARACTERS SO THAT WHEN WE DO OUR WHERE CLAUSE AND OUR CONDITIONAL IF WE SHOULD SPLIT OR NOT. IF WE SHOULD SPLIT WE GRAB IT FROM THE CHARACTERS WE’VE ALREADY SPLIT. IF NOT WE GRAB IT FROM THE TOKENS WE USED WHEN WE TOKENIZED. HERE WE DO A LITTLE REFORMATTING HOW THE SHAPE LOOKS. ONCE WE’VE DONE THAT AND WORD PIECE TOKENIZE AND COMBINE IT WITH THE VEHICLE VOCABULARY TABLE. WE GET RID OF THAT. LFRT. MADE IT THROUGH THAT. WASN’T TOO BAD. SO YOU KNOW WE APPLY THIS JUST AS WE DID BEFORE. WE HAVE A DATASET WE’VE CREATED WITH TF DATA. MAP OUT OUR PRE-PROCESSING ACROSS FROM THAT AND HERE WE CAN GRAB A CLASSIFIER BURN MODELS FROM THE OFFICIAL BURN MODELS AND TRAIN ON THAT CLASSIFIER. SO I KNOW THAT WAS A LOT TO GO THROUGH. HOPEFULLY YOU FOLLOWED ALONG. THE MAIN THING TO KNOW IS THAT AS TF.TEXT WE’RE LOOKING TO BASICALLY BRING IN ALL THAT PRE-PROCESSING INSIDE THE GRAPH SO YOU DON’T HAVE A PROBLEM. YOU DON’T HAVE TO WORRY ABOUT TRAINING SKEW, YOU CAN JUST WRITE YOUR TENSORFLOW AND TRAIN. SO WE DO THAT BY GIVING YOU WHAT I CONSIDER SUPERIOR DATA STRUCTURE FOR SEQUENCE DATA THROUGH RAGGED TENSORS AND APIs REQUIRED FOR THE PRE-PROCESSING. AGAIN YOU CAN INSTALL IT WITH TENSORFLOW TEXT. HERE ARE LINKS. GET HUB. IF THERE IS ANYTHING THAT YOU THINK IS MISSING THAT WE SHOULD ADD, FEEL FREE TO ADD AN ISSUE AND WE ALSO HAVE A TUTORIAL ON TENSORFLOW.ORG THAT YOU SHOULD CHECK OUT AND WALK THROUGH IT MORE SLOWLY FOR THE TOTAL LAB FOR YOU. THANKS. [APPLAUSE]>>WE STILL HAVE ABOUT SEVEN MINUTES LEFT SO WE CAN OPEN THIS UP FOR Q & A. THERE ARE SOME MICROPHONES ON EITHER SIDE BUT I CAN ALSO HELP PROVIDE THOSE IF NEEDED.>>DO YOU WANT TO GO THROUGH THE MICROPHONE? GO AHEAD.>>NICE TO SEE YOU HAVE ALL THIS SUPPORT. A QUICK QUESTION. DOES — CAN TF.TEXT HANDLE JAPANESE TEXT THAT’S A MIXTURE ALL THROWN TOGETHER?>>IN THE PREVIOUS EXAMPLE WHERE WE GO THROUGH THE CHARACTERS, THERE IS A LOT OF THE CORE UNICODE WE’VE ADDED TO CORE TENSORFLOW BUT WHEN — I DON’T KNOW WHERE WE ARE IN HERE. SO WHEN WE’RE SEARCHING FOR LIKE THE SCRIPT, THIS IS ICU, LIKE THE OPEN SOURCE LIBRARY SCRIPT AND SO YOU CAN JUST AS WELL LIKE GRAB THAT AND THEY HAVE DIFFERENT SCRIPTS.>>THANK YOU.>>WE’LL SWITCH THAT BACK AND FORTH.>>THANKS FOR THE INFORMATION. FOR INFERENCEING DO YOU SEND IN TEXT OR DO YOU HAVE TO PUT IT IN A TENSOR AND SEND IT?>>NO, YOU CAN SEND TEXT. AT INFERENCE TIME. IN THE TRAINING WE USE THE PRE-PROCESSING FUNCTION. YOU CAN USE THAT SAME PRE-PROCESSING FUNCTION WHEN YOU SAVE YOUR MODEL YOU GIVE IT A SERVING INPUT FUNCTION THAT BASICALLY DOES PRE-PROCESSING ON YOUR INPUT AND SO IF YOU SEND IN LIKE THOSE FULL STRING SENTENCES YOU CAN USE THE SAME FUNCTION OR A VARIATION OF THAT IN THAT INPUT FUNCTION AND THE PROCESS.>>THANK YOU VERY MUCH. MY QUESTION KIND OF RELATES TO HIS QUESTION SO WHAT IS THE ADVANTAGES OF APPLYING IT WITH A MAP VERSUS HAVING A — THAT DOES IT? EVEN WITH THE NEW RFC FOR PRE-PROPER SISING LAYER THAT DOITS.>>A LAYER THAT DOES THE PRE-PROCESSING?>>YES AND THEN IT’S SAVED AS PART OF THE MODEL.>>YOU COULD CERTAINLY DO SOME OF THIS WITH LAYERS. ACTUALLY, YOU KNOW, WE’RE LOOKING AT WHAT LAYERS WE SHOULD PROVIDE AND SOME OF OUR TEAM IS HELPING KAIROS BUILDING OUT THEIR PRE-PROCESSING LAYER AND IF THERE IS ADDED FUNCTIONALITY PEOPLE NEED WE’LL SUPPLY IT IN OUR TENSORFLOW TECH LIBRARY. IT’S UP TO YOU HOW YOU WANT TO APPLY THOSE.>>THANK YOU FOR THE TALK. TWO QUICK QUESTIONS. THE FIRST ONE IS DO WHAT YOU PROVIDE HAVE DECODING FUNCTIONS TO GO FROM THE TOKENS — FROM THE — TO THE SEQUENCE OF TEXT? YEAH. THE TF STREAM –>>FOR INSTANCE, IF YOU DECODE THEM WITH A BERT VOCABULARY YOU WILL HAVE ALL THIS ADDITIONAL CHARACTERS THERE AND THEN YOU WILL WANT TO GIVE THEM INTO A SEQUENCE OF TEXT, RIGHT?>>SO I THINK WHAT YOU ARE ASKING IS SO IF YOU SEND YOUR WORD THROUGH A BERT MODEL AND YOU GET A VECTOR REPRESENTATION YOU CAN TRANSFER IT BACK TO TEXT?>>THAT’S THE CASE WHEN YOU PRINT AND GENERATE TEXT. YOU MAY WANT TO MAP THE TEXT BACK, THE SEQUENCE OF TOKENS BACK INTO A STRING OF TEXT.>>THIS IS MORE ALONG THE LINES OF LIKE I THINK YOU WANT ENCODER/DECODER MODEL. THERE ARE MODELS THAT DO THIS. IT IS NOT SOMETHING THAT IS LIKE WE PROVIDE INSIDE THE LIBRARY.>>IT IS SUCH A DIFFERENT QUESTION. THE SECOND QUESTION IS WHY THIS IS NOT IN TENSORFLOW.>>SO WITH 2.0 YOU MIGHT KNOW THAT TENSORFLOW HAS GOTTEN RID OF CON TRIB AND FOR THE CORE TEAM HAS GOTTEN — IT’S TOO MUCH TO HANDLE THE TESTS ARE RUNNING TOO LONG AND IT IS REALLY TOO MUCH FOR ONE TEAM TO MAINTAIN. AND SO I THINK WE’LL SEE MORE KIND OF MODULAR TENSORFLOW TEXT THAT ARE FOCUSED ON ONE PARTICULAR AREA LIKE THIS. AS A TEAM WE WANT TO MAKE IT EASIER ACROSS SO A LOT OF STUFF WE’VE DONE WITH RAGGED TENSORS AND SOME OF THE STRING OPS ARE INCORE TENSORFLOW. FOR THOSE THINGS OUTSIDE THE SCOPE, LIKENED GRAMS AND TOKENIZATION, IT IS A SEPARATE MODULE.>>THANK YOU.>>SO SINCE TF.TEXT CAN BE INCORPORATED INTO THE TENSOR GRAPH IS IT INTENDED YOU BUILD A MODEL WITH THE PRE-PROCESSING STEP AND IF TRUE, WHAT ARE THERE PERFORMANCE IMPLICATIONS AND TENSORFLOW SERVING IF THAT’S THERE? HAS THAT BEEN MEASURED?>>YEAH. THERE IS DEFINITELY LIKE SOME PERFORMANCE. I THINK, YOU KNOW, IT’S DONE AT THE INPUT LEVEL. LET’S GET THAT TO HERE. IT’S DONE AT THE INPUT LEVEL. THIS IS ACTUALLY THE PROBLEM DATA IS LOOKING AT AS FAR AS CONSUMING THIS DATA AND THEN PARALLELIZING THESE INPUT FUNCTIONS AND SO IF YOU’RE ACTUALLY YOUR MODEL IS ON GPU OR CPU THE INPUT IS PARALYZED AND YOU ARE FEEDING AS MUCH DATA AS POSSIBLE. THIS IS SOMETHING YOU MIGHT WORRY ABOUT AND LOOK AT BUT IT IS ALSO WHAT A LOT OF THOSE PEOPLE ARE LOOKING.>>OKAY. I GUESS IF IT’S PART OF TENSORFLOW GRAPH HOW ARE LIKE THE THEY ARE COMPUTED — IT’S PRE-PROPER SENTION ON THE CPU?>>MOST IS DONE ON THE CPU. I SHOULD SAY ALL OF IT.>>QUICKLY ASK, ARE YOU COMPATIBLE WITH TF2. I JUST INSTALLED TENSORFLOW TEXT AND IT UNINSTALLED TENSORFLOW 2 AND INSTALLD 1.14.>>SO LIKE I SAID, THE VERSIONS NEED TO MATCH. IF YOU JUST INSTALL — (?) WHICH I THINK MAYBE WHY IT DID THAT IS BECAUSE THAT VERSION IS ACTUALLY A RELEASE CANDIDATE TO DO RT ZERO IT WILL REINSTALL TENSORFLOW FOR YOU. LOTS OF QUESTIONS.>>YES. FIRST OF ALL I WOULD LIKE TO SAY THIS IS REALLY COOL. SECOND IS THAT DO TF.TEXT INTEGRATE WITH OTHER LIBRARIES OR ANYTHING IN THAT AREA OUT OF CURIOSITY?>>NO, OUR FOCUS WAS REALLY ON TENSORFLOW RIGHT NOW.>>OKAY.>>THANK YOU, ROBBY. ♪>>HEY, EVERYBODY WELCOME TO THE AO2019. SO IN THE KEYNOTE WE STILL HAVE SOME AMAZING STUFF YOU CAN ACCOMPLISH WITH ON-DEVICE ML AND I’M HERE WITH TENSORFLOW LIFE SHOWING DEVELOPERS HOW THEY CAN BOULD THAT STUFF THEMSELVES. YOU HAVE EVERYTHING FROM OBJECT DETECTION TO IMAGE CLASSIFICATION AND VOICE RECOGNITION AND WE’RE SHOWING PEOPLE HOW THEY CAN DEPLOY THOSE DOWN TO DEVICES THAT ARE REALLY TINY AND RUN ON A SINGLE BATTERY. SO TO SHOW YOU WHAT’S ON TENSORFLOW LIFE WE’VE BUILT THIS AMAZING EXPERIENCE CALLED DANCE LIKE.>>IT’S A SUPER FUN APPLICATION USING TENSORFLOW LIFE WHICH IS THE MOBILE EMBEDDED SYSTEMS FRAMEWORK FOR MACHINE LEARNING IT BASICALLY TEACHES PEOPLE HOW TO DANCE SO IT DOES THIS BY POST SEGMENTATION ON THE GBU. WE HAVE A — WE HAVE A BUNCH OF GPU FOR TENSORFLOW LIVE SO YOU SHOULD CHECK THOSE HOW THE IT ENABLES TO YOU RUN SUPER FAST MODELS AND WE’RE RUNNING 2 AND WE’RE GIVING HELPFUL FEEDBACK TO HELP THEM DANCE BETTER AND, YOU CAN GO TO TENSORFLOW.ORG/LIVE. THERE’S HEAPS OF TUTORIALS AND YOU CAN GO AND BUILD AN AMAZING APPLICATION LIKE DANCE LIKE AND RELEASE TO THE WORLD. ♪>>WHAT WE’RE LOOKING AT HERE IS A MODEL CALLED POSE NET WHICH IS TRAINED ON IMAGES OF HUMAN BEINGS AND IT TRAINED TO PREDICT THEIR SKELETON POST AND THIS ONE IS RUNNING ENTIRELY IN THE BROWSER IS A LIBRARY CALLED TENSORFLOW.JS IS A MACHINE LEARNING LIBRARY FORAVA SCRIPT THAT CAN RUN ENTIRELY IN THE BROWSER AND USE YOUR GPU THROUGH WEB GL, AND IT ALSO RUNS IN NO JS USING THE ENTIRE TENSORFLOW C++ BINARY AND ALL THE HARDWARE STORIES AND PIANO GENI IS A MODEL THAT’S RUN ENTIRELY IN THE BROWSER AS WELL THAT GENERATES MINI NOTES AS PART OF BROWSER AND IT LETS THE HUMAN CONDITION WHERE THE MODEL IS ACTUALLY GOING TO GO. TENSORFLOW JOHNSON COUNTY S IS OPEN SOURCE AND IS AVAILABLE ONLINE ON TENSORFLOW.ORG JS. HEAD OVER THERE TO START BUILDING NEW MACHINE LEARNING FOR THE BROWSER. ♪ TENSORFLOW L. TESTIMOMBE GIVEN HOW LOUD AND H NOISY AND COMPLEX ITS. WE’RE USING AN OPEN SOURCE MACHINE LEARNING TOOL TENSORFLOW THAT’S ABLE TO DETECT A LOGGING TRUCK OR THE SOUNDS OF BIRDS OR CHAINSAWS IN THE FOREST IT’S AMAZING THAT THEY CAN FIND AND SEND A REAL TIME TO THE RANGERS TO THE GUARDS TO THE CHIEFS SO THEY CAN DECIDE HOW TO RESPOND. . .>>THEIR ENGINEERS ON OUR TENSORFLOW TEAM AT GOOGLE AND THEY’RE GOING TO TALK TO US ABOUT PERFORMANCE OF SCALING AND TENSORFLOW 2.0.>>THANKS. [APPLAUSE]>>SO I’M TAYLOR. I’M AN ENGINEER ON THE CARA’S TEAM.>>I’M PREPRIORA AND I’M ON THE TENSORFLOW TEAM.>>AND TODAY WE’RE GOING TO BE TALKING WRITING PERFORMANCE MODELS IN TENSORFLOW 2.2. JUST TO GIVE A BRIEF OUTLINE OF THE TALK WE’LL TALK ABOUT THE STRUCTURE AND TALK ABOUT THE DIFFERENT APIS THAT YOU CAN USE TO MAKE YOUR TRAINING MORE PERFORMANT.pTHE FIRST API WE’RE TALKING ABOUT IS TF DATA WHICH IS WRITING HIGH PERFORMANCE PIPELINES TO AVOID VARIOUS SORTS OF STALLS AND MAKE SURE THAT YOUR TRAINING ALWAYS HAS DATA AS IT’S READY TO CONSUME IT. NEXT WE’RE GOING TO BE TALKING OF TF FUNCTION. THIS IS HOW CAN YOU CAN FORMULATE YOUR TRAINING IN A WAY YOU’RE EFFICIENT SO THE RUN TIME CAN BE RUN WITH VERY LITTLE OVERHEAD, AND FINALLY WE’RE GOING TO BE TALKING TF DISTRIBUTE WHICH IS HOW TO TARGET YOUR TRAINING TO A SPECIFIC HARDWARE AND ALSO SCALE OUT TO MORE HARDWARE. IN ORDER TO DEMONSTRATE ALL OF THESE APIS WE’RE GOING TO BE WALKING THROUGH A CASE STUDY STARTING WITH THE MOST NAIVE IMPLEMENTATION AND THEN PROGRESSIVELY ADDING WE’LL HAVE MORE PERFORMANT APIS AND LOOKING AT HOW THAT WILL BE TRAINING. WE’LL BE TRAINING A CLASSIFIER BECAUSE WE’RE AN INTERNET COMPANY THAT’S WHAT WE DO AND WE’LL BE ON MOBILE NET V2 WHICH IS LARGELY CHOSEN ARBITRARILY JUST AS A REPRESENTATIVE TOOL SO ITSHOULD BE APPLICABLE TO YOUR WORKFLOWS AS WELL. AND THERE’S A NOTEBOOK AND I’LL PUT THE LINK AT THE END OF THE SLIDE SO YOU’RE ENCOURAGED TO TAKE A LOOK AFTERWARD AND COMPARE BACK TO THE TALK.>>A BRIEF INTERVIEW OF TRAINING LOOPS AND TENSORFLOW 2.2. WE WANT TO LOOK AT THE MECHANICS OF WHAT THE SYSTEM IS ACTUALLY DOING AND HOW WE CAN MAKE IT PERFORMANT. IF YOU’RE USED TO USING A LIAR LEVEL API LIKE CARA’S MODEL FIT THESE LESSONS ARE STILL BROADLY APPLICABLE. IT’S JUST THAT SOME OF THIS WILL BE DONE AUTOMATICALLY FOR YOU. SO IF WE WANT TO SHOW WHY THESE APIS IS IMPORTANT IT’S USEFUL TO REALLY DIG INTO THE INTERNALS OF THE TRAINING LOOM. SO BECAUSE WE’RE TRAINING ON MOBILE NET WE’VE CHOSEN TO USE THE CARIS APPLICATIONS MOBILE NET V2 WHICH IS A CANNED IMPLEMENTATION WE’RE TRAINING A CLASSIFIER SO WE’RE GOING TO BE USING SIGMOID CROSS-LOGICS WHICH IS A STABLE CROSS-FUNCTION AND THEN WE’RE GOING TO BE USING THE CARIS SGD OPTIMIZER. SO IF WE WERE TO CALL OUR MODEL ON OUR FEATURES IN THIS CASE IT’S AN IMAGE CLASSIFICATION TASK SO OUR FEATURES ARE AN IMAGE, WE WOULD GET LODGIGS WHICH IS THE PREDICTIVE CLASS AND IF WE PREPARE OUR LOSS FUNCTIONS IN THE LABELS IN THE LOGIC IT GIVES US A SCALE SCALAR LOSS TO UPDATE OUR MODEL. WE WANT TO CALL ALL OF THIS UNDER A TF GRADIENT TABLE WHICH WATCHES THE COMPUTATIONS AS THEY’RE PERFORMED AND CONTAINS THE METADATA AND KEEPS THE ACTIVATIONS ALIVE WHICH ALLOWS US TO TAKE GRADIENTS WHICH MAY CAN PASS TO THE OPTIMIZER TO UPDATE OUR MODEL WEIGHTS, AND FINALLY IF WE WRAP ALL OF THIS INTO A STEP AND ITERATE OVER OUR DATA IN MINI BATCHES LOGGING THIS STEP IN EACH MINI BATCH THIS COMPRISES THE GENERAL STRUCTURE OF OUR TRAINING LOOP. AND WE’LL LOOK AT THE PERFORMANCE A LITTLE BIT LATER. WHAT DO WE WANT TO DO WE WANT TO SHUFFLE THE DATA SO WE SEE A DIFFERENT ORDER IN EACH EPOCH. WE’LL NEED TO LOAD ALL THE IMAGES FROM DISK INTO MEMORY SORT OF CAN BE CONSUMED BY THE TRAINING PROCESS. WE’LL ALSO WANT TO DO SOME AUGMENTATION SO WE’LL WANT TO RESIZE THE IMAGES FROM THEIR NATIVE FORMAT BACK TO THE FORMAT THAT THE MODEL EXPECTS AND THEN WE’LL DO SOME TASK-SPECIFIC OPTIMIZATION SO IN THIS CASE WE’LL RANDOMLY FLIP THE IMAGES AND WE’LL ADD SOME PIXEL-LEVEL NOISE TO MAKE OUR TRAINING A LITTLE BIT MORE ROBUST. WE’LL NEED TO BATCH THESE AND WE’LL NEED TO COLLECT THEM INTO MINI BATCHES AND THE KEY OPERATION FOR THAT BATCHING OPERATION IS A CACAT NATION. IT’S WORTH NOTING THAT DIFFERENT PARTS OF THE DATA PIPELINE WILL STRESS DIFFERENT PARTS OF THE SYSTEM. SO FOR LOADING THE DISK THIS IS AN IOBOUND TASK AND WE’LL GENERALLY WANT TO CONSUME THIS I/OS AS FAR AS POSSIBLE SO WE’RE NOT CONSTANTLY WAITING FOR IMAGES TO ARRIVE FROM DISK 1 AT A TIME. SECONDLY, THE AUGMENTATIONS TENDS TO BE RATHER CPU INTENSIVE ‘CAUSE WE’RE DOING VERY SORTS OF MATH IN OUR AUGMENTATION AND BATCHING SEEMS TO BE A MEMORY INTENSIVE TASK BECAUSE WE’RE COPYING EXAMPLES FROM THEIR ORIGINAL LOCATION INTO THE MEMORY BUFFER INTO OUR MINI BATCH. NOW THAT WE HAVE A SCAFFOLDOLDING LET’S RUN IT. WE’LL BE RUNNING IT IN A NVIDIA GPU, AND TENSORFLOW WILL INTEND TO PLACE OPS ON THE OPTIMAL DEVICE AND WE’RE DOING LARGE MATRIX APPLICATIONS IT WILL PLACE THEM ON THE GPU. HOWEVER, WE FIND OUR TRAINING AT THE START IS PRETTY LACKLUSTER WE’RE UNDER 100 IMAGES A SECOND AND IF WE ACTUALLY LOOK AT OUR DEVICE UTILIZATION IT’S WELL UNDER 20% SO WHAT’S GOING ON HERE. WE CAN ACTUALLY DO SOME PROFILING AND TENSORFLOW AND TENSORFLOW BOARD MAKES THIS QUITE EASY TO DO. YOU MIGHT ALREADY BE FAMILIAR WITH TENSOR BOARD WHICH IS THE STANDARD WAY OF MONITORING TRAINING AS IT RUNS SO YOU MIGHT BE FAMILIAR WITH THE SCALARS TABITHA SHOWS WATCHES AND METRICS AS YOUR TRAINING CONTINUES AND THERE’S A NEW TAB WHICH IS THE PROFILER TAB AND WHAT THIS DOES IS IT LOOKS AT EACH OP AS IT RUNS IN TENSORFLOW AND SHOWS YOU A TIMELINE OF HOW YOUR TRAINING IS PROGRESSING AND WE’RE GOING TO LOOK IN A LITTLE BIT MORE DETAIL ABOUT HOW TO READ AND INTERPRET THESE TIMELINES. BEFORE WE DO, HOWEVER, LET’S TALK ABOUT HOW TO ENABLE IT. SO IT’S QUITE STRAIGHTFORWARD. YOU SIMPLY TURN ON THE PROFILER, THIS WILL SIGNAL TO THE RUN TIME THAT IT SHOULD KEEP RECORDS OF OPS AS IT RUNS THEM. WRITE YOUR PROGRAM AS NORMAL AND THEN SIMPLY EXPORT THIS TRACE AND THIS WILL FUTURE IN A FORMAT THAT TENSOR BOARD CAN CONSUME AND DISPLDISPLAY FOR YOU. THE TOP IS THE CPU AND THE BOTTOM IS OF THE VARIOUS THREADS AND WE’RE LOOKING AT 3 DIFFERENT STEPS. EACH VERTICAL LINE HAS BEEN RUN BY TENSORFLOW. SO YOU CAN SEE THAT THE OPS ARE SCHEDULED FROM THE CPU AND THEN THEY APPEAR ON THE GPU STREAM AND THE GPU ACTUALLY CONSTITUTES THE OPS TO WHERE THE HEAVY LIFTING IS DONE BUT YOU CAN ALSO SEE THAT WE HAVE THESE LARGE GAPS IN BETWEEN WHERE NO OPS ARE RUNNING AND OUR SYSTEM IS ESSENTIALLY STALLED AND THIS IS BECAUSE AFTER EACH STEP WE HAVE TO WAIT FOR OUR NAIVE GENERATOR TO PRODUCE THE NEXT BATCH OF DATA SO THIS IS, OBVIOUSLY, QUITE INEFFICIENT. SO PRIA IS GOING TO TELL YOU HOW WE CAN DO THAT.>>THANKS, TAYLOR. SO AS TAYLOR JUST SHOWED US, WRITING YOUR OWN INPUT PIPELINE CAN BE VERY INSUFFICIENT. TENSORFLOW APPLIES THE TENSORFLOW 2.2 TO APPLY SCALABLE INPUT PIPELINES. YOU CAN THINK OF THE PIPELINE AS AN ETL PROCESS SO THE FIRST STAGE IS THE EXTRACTS STAGE WHERE WE READ THE DATA FROM — LET’S SAY A NETWORK STORAGE OR FROM YOUR LOCAL DISK AND THEN YOU’RE PARSING THE FORMAT. THE SECOND STAGE IS THE TRANSFORM STAGE WHICH IS WHERE YOU TO TAKE TE INPUT FILE DATA AND TRANSFORM IT INTO A FORM THAT’S AMENABLE TO YOUR COMPUTATION SO THIS COULD BE IMAGE TRANS FORMATIONS AND SPECIFIC TO YOUR ML TASK SORE THEY COULD BE GENERIC PARSING FOR YOUR TASK. ONCE YOU TRANSFORM IT YOU CAN ADD IT INTO I DON’T AXILATER FOR — INTO YOUR AXILATER, AND LET’S PART WHAT TAYLOR SHOWED YOU BEFORE IN A MUCH MORE EFFICIENT WAY, AND THERE’S A NUMBER OF STEPS AND I’LL GO OVER THEM ONE BY ONE. THE FIRST ONE IS WE CREATE A SIMPLE DATASET CONSISTING OF ALL THE FILE NAMES IN OUR INPUT SO IN THIS CASE WE HAVE A NUMBER OF IMAGE FILES AND THE PATH GLOCK SPECIFIES HOW TO FIND THESE FILE NAMES. THE SECOND THING WE DO IS WE WANT TO SHUFFLE THIS LIST BECAUSE IN TRAINING YOU WANT TO SHUFFLE THE INPUT DATA AS IT’S COMING IN AND WE APPLY OUR TRANS FORMATION CALLED THE MAP TRANS FORMATION AND WE PROVIDE THE GET BYTES LABEL CUSTOMIZATION AND WHAT IT DOES IT’S GOING TO READ THE FILE ONE BY ONE USING THE TFI READ FILE API AND IT USES THE FILE NAME PATH TO COMPUTE THE LABEL AND RETURNS BOTH OF THESE. SO THESE 3 STEPS THAT I TALKED ABOUT. WE’RE READING THE FILE AND PARSING THE DATA. NOTICE THIS IS ONE WAY IN WHICH YOU CAN READ DATA AND THERE ARE A NUMBER OF APIS THAT YOU CAN USE FOR OTHER SITUATIONS WHICH WE HAVE LISTED HERE. THE MOST NOTABLE ONE IS THE TF RECORD DATASET API WHICH YOU WOULD USE IF YOU ARE DATA IS IN THE TF FILE RECORD FORMAT AND THIS IS THE MOST PERFORMANT FILE FORMAT. IF YOU HAVE YOUR INPUT IN MEMORY LET’S SAY YOU CAN USE THE FROME TENSOR SLICES AND USE THAT SEQUENCE TRANS FORMATIONS JUST LIKE THIS. THE NEXT THING WE DO ANOTHER MAP TRANS FORMATION TO NOW TAKE THIS RAW IMAGE DATA BUT CONVERT IT AND DO SOME PROCESSING TO MAKE IT AMENABLE FOR OUR TRAINING TASKS. SO WE HAVE THIS PROCESS IMAGE FUNCTION HERE WHICH WE PROVIDE TO THIS MAP TRANS FORMATION AND IF YOU TAKE A LOOK THIS COULD LOOK SOMETHING LIKE THIS. WE HAD SOME DECODING OF THE IMAGE DATA AND THEN WE APPLY SOME IMAGE PROCESSING TRANS FORMATIONS SUCH AS RESIZINGS FLIPPING NORMALIZATION AND SO ON. THE KEY THINGS TO NOTE HERE THESE TRANS FORMATIONS ARE ACTUALLY VERY SIMILAR AND CORRESPOND 1:1 TO WHAT YOU SAW IN THE PYTHON VERSION BUT THE BIG THING WE’RE USING TENSORFLOW OPS TO DO THOSE TRANS FORMATIONS. THE SECOND TF DATA WILL TO TAKE THE CUSTOM FUNCTION THAT YOU SPECIFY TO THE MAP TRANS FORMATION AND IT WILL TRACE IT IN A GRAPH AND RUN IT IN THE C++ TIME INSTEAD OF HIGH TIME AND THIS IS WHAT ALLOWS IT TO MAKE IT MUCH MORE EFFICIENT THAN PYTHON. AND FINALLY WE USE THE BATCH TRANS FORMATION TO PATCH THE ELEMENTS OF YOUR INPUT INTO MINI BATCHES AND THIS IS A VERY COMPACT FOR TRAINING EFFICIENCY IN ML TASKS. SO BEFORE I HAND IT OFF TO TAYLOR TO TALK ABOUT HOW USING TF DATA CAN IMPROVE THE PERFORMANCE ON A MOBILE NET EXAMPLE, I WANT TO WALK THROUGH A ADVANCED PERFORMANCE OPTIM OPTIMIZATION I’LL GO THROUGH THEM QUICKLY AND YOU CAN READ ABOUT THEM IN MUCH MORE DETAIL ON THE PAGE THAT’S LISTED HERE. THE FIRST OPTIMIZATION THAT I WANTED TO TAKE ABOUT IS PIPELINING AND THIS IS THE CONCEPTUALLY VERY SIMILAR TO ANY OTHER SOFTWARE PIPELINE THAT YOU MIGHT BE AWARE OF. THE HIGH-LEVEL IDEA HERE IS THAT WHEN YOU’RE TRAINING A SINGLE BATCH OF DATA ON YOUR ACCELERATOR WE WANT TO USE THE CPU RESOURCE TO PREPARE THE NEXT BATCH OF DATA. WHAT THIS WILL DO WHEN THE NEXT TRAINING STARTS WE DON’T HAVE TO WAIT FOR THE NEXT BATCH OF DATA TO BE PREPARED. IT WILL AUTOMATICALLY BE PREPARED AND IT REDUCES THE OVERALL WAITING TIME SIGNIFICANTLY. YOU CAN USE THE PREFIX TRANS FORMATION AS SHOWN HERE. THE SECOND OPTIMIZATION THAT I WANT TO TALK ABOUT IS PARALLELIZING THE TRANSFORMATION AND APPLY TO EACH ELEMENT TO THE DATA INPUT IN EACH SEQUENCE. BUT IF THERE’S PARALLELING THERE’S NO REASON TO DO IT IN SEQUENCE, RIGHT, SO YOU CAN PARALLELIZE ARGUMENT TO THE MAP TRANS FORMATION THAT IT SHOULD RUN THESE MAP OPERATIONS ON YOUR ELEMENTS OF THE DATASET IN PARALLEL. AND THE THIRD THING I WANT TO TALK ABOUT IS THE PARALLELIZATION STATE AND PROVIDING YOUR ELEMENTS IN SEQUENCE CAN BE SLOW SIMILARLY READINFILES ONE BY ONE CAN BE SLOW AS WELL. SO, OF COURSE, YOU WANT TO PARALLELIZE IT AND IN THIS EXAMPLE SINCE WE ALSO ARE — SINCE WE’RE USING A MAP TRANS FORMATION TO READ OUR FILES IS VERY SIMILAR TO WHAT WE JUST SAW.TRANSFORMATION TO READ OUR S IS VERY SIMILAR TO WHAT WE JUST SAW. AND YOU ADD AN ELEMENT TO READ YOUR FILES. NOTE THAT IF YOU’RE USING ONE OF THE BUILDING FILE READERS YOU CAN PROVIDE A VERY SIMILAR SET TO PARALLELIZE THE SIGNALING THERE. SO IF YOU’VE BEEN PAYING CLOSE ATTENTION YOU’LL NOTICE WE HAVE THESE MAGIC NOTES ON X, Y, Z AND TRY AND DETERMINE THE OPTNEL VALUE OF THESE. IF YOU SET THEM TOO LOW YOU MIGHT NOT BE USING ENOUGH PARALLELISM IN YOUR SYSTEM AND IF YOU SET THEM TOO HIGH, IT MIGHT BE TO CONTENTION AND ACTUALLY HAVE THE OPPOSITE EFFECT OF WHAT YOU WANT. FORTUNATELY, TF DATA REALLY MAKES IT EASY FOR YOU TO SPECIFY THESE AND INSTEAD OF SPECIFYING SPECIFIC VALUES FOR THESE X, Y, Z ARGUMENTS, YOU CAN SIMPLY USE THIS SCANT EXPERIMENTAL AUTOTUNE AND WHAT IT DOES TO THE TF DATA RUN TIME IT SHOULD DO THE AUTOTUNING FOR YOU AND DETERMINE THE OPTIMAL VALUES FOR THESE ARGUMENTS BASED ON YOUR WORKLOAD, YOUR ENVIRONMENT, YOUR SETUP AND SO ON. SO THAT’S ALL FOR TF DATA AND I’LL HAND IT OFF TO TAYLOR TO SEE WHAT KIND OF PERFORMANCE BENEFITS YOU CAN SEE.>>THANKS, PRIA. ON THE RIGHT HERE WE CAN SEE THE TIMELINE BEFORE AND AFTER WE ADD TF DATA SO BEFORE WE HAVE THIS LONG STALL BETWEEN THIS TRAINING STEPS WHILE WAITING FOR DATA, ONCE WE’VE USED TF DATA YOU’LL NOTICE TWO THINGS ABOUT THIS TIMELINE AFTER. THE FIRST IS THAT THE TRAINING STEP BEGINS IMMEDIATELY AFTER THE PREVIOUS ONE. IT WAS PREPARE THE PREVIOUS BATCH WHILE THE TRAINING STEP WAS HAPPENING, AND THE SECOND STEP BEFORE IT ACTUALLY TOOK LONGER THAN THE TIME OF A BATCH IN ORDER TO PREPARE THE NEXT BATCH; WHEREAS, NOW WE SEE NO LARGE STALLS IN THE TIMELINE. THE REASON IS TF.DATA — BECAUSE OF THE NATIVE PARALLELISM CAN NOW PRODUCE BATCHES MUCH MORE QUICKLY THAN THE TRAINING CAN CONSUME THEM. AND YOU CAN SEE THIS MANIFEST IN OUR THROUGH-PUT WITH MORE THAN A 2X IMPROVEMENT. BUT SOMETIMES THERE’S EVEN MORE STALLS. IF WE ZOOM ON THE TIMELINE OF ONE OF OUR TRAINING STEPS, WE’LL SEE THAT THERE ARE A NUMBER OF THESE VERY SMALL GAPS AND THESE ARE LAUNCHOVER HEADS. SO IF WE FURTHER LOOK AT DIFFERENT PORTIONS OF THE GPU STREAM, NEAR THE END THIS IS WHAT A HEALTHY PROFILE LOOKS LIKE. YOU HAVE ONE OP ONES AND THEN FINISHES AND IMMEDIATELY, THE NEXT OP CAN BEGIN AND THIS RESULTS IN AN EFFICIENT AND SATURATED ACCELERATOR. ON THE OTHER HAND, IN SOME OF THE EARLIER PARTS OF THE MODEL, THE GPU IMMEDIATELY CHEWS THROUGH IT AND IT WAITS IDLE FOR PYTHON TO COULD YOU THE NEXT OP WHICH LEADS TO A VERY POOR ACCELERATOR UTILIZATION AND EFFICIENCY. SO WHAT CAN WE DO? WELL, IF WE GO BACK TO OUR TRAINING STEP, WE CAN SIMPLY ADD THE TF FUNCTION DECORATOR. WHAT THIS WILL DO IT WILL TRACE THE ENTIRE TRAINING STEP INTO A TF GRAPH WHICH CAN BE RUN VERY EFFICIENTLY FROM THE TF RUN TIME AND THIS IS THE ONLY CHANGE NEEDED AND IF WE NOW LOOK AT OUR TIMELINE WE SEE ALL OF THE SCHEDULING IS DONE IN THIS SINGLE, MUCH FASTER OP ON THE CPU AND THEN THE WORK APPEARS ON THE GPU AS BEFORE AND THIS ALSO ALLOWS US TO USE THE RICH LIBRARY OF GRAPH OPTIMIZATIONS THAT WERE DEVELOPED IN TENSORFLOW 1. THIS IS ALMOST ANOTHER 23567K9 OF 2 IMPROVEMENT AND IT’S PRETTY OBVIOUS ON THE TIMELINE ON THE RIGHT Y. WELL, BEFORE ON TF FUNCTION WE WERE WAITING EARRINGERLY WE HAD ALL THE GRAPHS WAITING FOR ON THAT AND ONCE IT’S LAUNCHED ON THE C++ RUN TIME IT’S ABLE TO DO A MUCH BETTER JOB WITH KEEPING UP WITH THE GPU. SO THE NEXT OPTIMIZATION THAT I MEAN, I GOT TO TO TALK ABOUT IS XLA WHICH STANDS FOR ACCELERATOR LINEAR ALABRA. WE HAVE JUST THE SIMPLE EXAMPLE OF A GRAPH WITH SOME SKIP CONNECTIONS. WHAT IT WILL DO IT WILL CLUSTER THAT GRAPH INTO SUBGRAPHS AND THEN COMPILE THE ENTIRE SUBGRAPH INTO A VERY EFFICIENT FUSED CONCERNAL AND THERE’S A NUMBER — KERNEL, AND THERE’S A GAIN FROM THESE USED KERNELS. YOU GET A MUCH MORE EFFICIENT MEMORY ACCESS BECAUSE THE KERNEL CAN USE THE DATA WHEN IT’S HOT IN THE CACHE AND THEN BRINGING IT DOWN TO THE HIGHARCHY BRINGING IT ALL BACK. IT REDUCES — THE LAUNCH OVERHEAD AND IT’S THE EXECUTIVE LAUNCH OVERHEAD BY RUNNING FEWER OPS WITH THE SAME MATH AND FINALLY XLA IS HEAVILY OPTIMIZED TO TARGET HARDWARE SO IT USES THING FOR VECTOR INSTRUCTIONS, SPECIALIZED ON SHAPES AND CHOOSE LAOUTS SO THAT THE HARDWARE IS ABLE TO HAVE VERY EFFICIENT ACCESS PARTNERS. AND IT’S QUITE STRAIGHTFORWARD TO ENABLE. YOU SIMPLY USE THIS TF CONFIG JET FLAG WHICH WILL CAUSE EVERY TF FUNCTION THAT RUNS TO ATTEMPT TO COMPILE AND RUN WITH XLA AND YOU CAN SEE THAT IN OUR EXAMPLE, IT’S A VERY STARK IMPROVEMENT. IT’S ABOUT A 2.5X IMPROVEMENT IN THROUGH-PUT. THE ONE CAVEAT WITH XLA THAT A LOT OF THE UTILIZATIONS IT USES IS SPECIALIZING ON SHAPES SO IT NEEDS TO RECOMPILE EVERY TIME IT SEES NEW SHAPES SO IF YOU HAVE A EXTREMELY DINAME SHAPE AND THE SHAPES ARE DIFFERENT IN EACH BATCH YOU WIND UP SPENDING MORE TIME ON THE XLA COMPILE THAN ON THE COMPUTATION YOU SHOULD TRY XLA BUT IF YOU SEE PERFORMANCE REGRESSION RATHER THAN PERFORMANCE GAIN IT’S LIKELY THAT’S THE REASON. SO NEXT WE’RE GOING TO TALK ABOUT MIXED PRECISION IF WE WANT TO GO EVEN FASTER WE CAN GPU SOME OF OUR NUMERIC STABILITY IN ORDER TO ATTAIN FASTER TRAINING PERFORMANCE. SO HERE I HAVE THE IEE IN TENSORFLOW FLOW BUT THERE’S ALSO TWO HALF PRECISION FORMATS THAT ARE IN PRECISION, AND THE OTHER IS FLOAT 16 WHERE WE GPU BOTH SUM EXPONENT IN EXCHANGE FOR KEEPING A LITTLE BIT MORE MAN TISSA AND WHAT’S IMPORTANT ABOUT THESE FORMATS THEY HAVE NATIVE HARDWARE SUPPORT. SO TPU HAS HARDWARE SUPPORT FOR B FLOAT OPERATIONS AND GPUS HAVE SUPPORT FOR EVERY EFFICIENT FLOAT 16, AND IF WE REDUCE THESE PERFORMANCE WE CAN GET HIGH SPEEDUPS FROM THE HARDWARE. IN ORDER TO ENABLE THIS WE NEED TO DO A COUPLE OF THINGS. FIRST WE NEED TO CHOOSE A LOSS SCALE SO WHAT IS A LOSS SCALE. A LOSS SCALE IS A CONSTANT THAT’S INSERTED INTO THE COMPUTATION WHICH DOESN’T CHANGE THE MATHEMATICS OF THE COMPUTATION BUT IT CHANGES THE NUMERICS. THIS IS A KNOB OF THE RUN TIME TO ADJUST THE COMPUTATION TO KEEP IT IN THE NUMERICALLY STABLE RANGE, AND NEXT WE’LL WANT TO SET A MIXED PRECISION POLICY AND THIS WILL TELL KE IT WILL CAST — KERAS TO MAKE SURE THE COMPUTERTION IS ACTUALLY REPRESENTING IN THE FLOATER POINT COMPUTATION AND WE’LL WANT TO WRAP OUR OPTIMIZER IN THIS LOSS SCALE AND THIS IS THE HOOK LOOP IN WHICH IT’S INSERTED SO AS TRAINING HAPPENS, TENSORFLOW WILL DO A DYNAMIC ADJUSTMENT OF THIS LOSS SCALE TO BALANCE VANISHING GRADIENTS FROM FP UNDERFLOW WHILE STILL PREVENTING NANS FOR FP13 OVERFLOW. IF YOU’RE WORKING FOR THE MODEL FIT WORKFLOW THIS IS WHAT WE NEED TO DO AND WE NEED TO DO THE MIXED PRECISION TRAINING WHICH MAKES COMPUTING A SINGLE EXAMPLE MUCH LESS EXAMPLE SO IF WE JUST TURN ON MIXED PRISON WE CAN GO FROM A SACHTION TO AN UNDERLOADED IN BATCH 16 AND WE CAN GO BACK TO FILLING ALL THE HARDWARE REGISTERS AND WE CAN SEE THIS IN OUR EXAMPLE. IF WE JUST TURN ON FLOAT 16 THERE’S ACTUALLY NO IMPROVEMENT IN PERFORMANCE BUT IF WE THEN INCREASE OUR BATCH SIZE, THEN WE SEE A VERY SUBSTANTIAL IMPROVEMENT IN PERFORMANCE. IT IS WORTH NOTING THAT BECAUSE WE BOTH REDUCED THE NUMERIC PRECISION AND CHANGED THE BATCH SIZE THIS CAN REQUIRE THAT YOU RETUNE THE HYPERPARAMETERS. AND THEN FINALLY IF WE LOOK AT WHAT ARE THE REMAINING BIT OF PERFORMANCE — BITS OF PERFORMANCE. 60% OF WHAT’S LEFT IS ACTUALLY THE COPY FROM THE HOST CPU TO THE GPU AND IN A LITTLE BIT PRIA WILL TALK A LITTLE BIT ABOUT DISTRIBUTION AND ONE OF THE THINGS THAT THE DISTRIBUTION AWARE CODE IN TENSORFLOW WILL DO WILL AUTOMATICALLY ATTACH THIS PREFIX SO YOU’LL ACTUALLY GET THE 60% FOR FREE AND FINALLY YOU HAVE HAND TUNING SO YOU CAN GPU SOME OF THE NUMERIC STABILITY. YOU CAN MESS WITH THREAD POOLS AND MANUALLY TRY TO OPTIMIZE LAOUTS WHICH IS INCLUDED FOR COMPLETENESS IN CASE YOU’RE CURIOUS WHAT REAL LOW-LEVEL HAND TUNING LOOKS LIKE BUT FOR MOST CASES, GIVEN THAT WE CAN GET THE VAST MAJORITY OF THE PERFORMANCE WITH JUST SIMPLE IDIOMATIC EASY TO USE APIS THIS VERY FINE HAND TUNING IS NOT RECOMMENDED. BUT ONCE WE’VE ACTUALLY SATURATED A SINGLE DEVICE, WE NEED TO DO SOMETHING ELSE TO GET ANOTHER ORDER OF MAGNITUDE IMPROVEMENT AND SO PRIA IS GOING TO TALK TO YOU ABOUT THAT.>>THANKS, TAYLOR. ALL RIGHT. SO TAYLOR TALKED ABOUT A NUMBER OF DIFFERENT THINGS YOU CAN DO TO GET THE MAXIMUM PERFORMANCE OUT OF THE A SINGLE OPINION WITH A SINGLE GPU. BUT IF YOU WANT YOUR TRAINING TO GO EVEN FASTER, YOU NEED TO START SCALING UP. SO MAYBE YOU WANT TO ADD MORE GPUS TO YOUR MACHINE OR PERHAPS YOU WANT TO TRAIN ON MULTIPLE MACHINES IN A CLUSTER OR MAYBE YOU WANT TO USED SPECIALIZED HARDWARE SUCH AS CLOUD TPUS. YOU ARE ALLOWED TO DO JUST THAT. WE BUILT THIS API WITH 3 KEY GOALS IN MIND. THE FIRST GOAL IS EASE OF USE. WE WANT USERS TO BE ABLE TO USE THE DISTRIBUTED API WITH VERY LITTLE CHANGES TO THEIR CODE. THE SECOND GOAL IS TO GIVE GREAT PERFORMANCE OUT OF THE BOX. WE DON’T WANT USERS TO CHANGE THEIR CODE OR TURN A LOT OF KNOBS TO GET THE MAXIMUM EFFICIENCIES OUT OF THEIR RESOURCES, AND WE WANT THE APIS TO WORK IN A VARIETY OF DIFFERENT SITUATIONS, AND IF YOU WANT TO SCALE OUT TO MULTIPLE GPUS OR MULTIPLE OPINIONS OR TPUS OR IF YOU WANT DIFFERENT DISTRIBUTED TRAINING ARCHITECTURE SUCH AS SYCHRONOUS OR ASYCHRONOUS TRAINING OR YOU’RE USING DIFFERENT TO MY APIS. MAYBE YOU’RE USING HELLO KERAS OR YOU HAVE A CUSTOM HARDWARE LOOP AS EXPLAINED EARLIER. YOU WANT YOUR DISTRIBUTION STRATEGY TO WORK IN ALL OF THESE POTENTIAL CASES. THERE ARE A NUMBER OF DIFFERENT WAYS IN WHICH YOU CAN USE THIS API AND WE’VE LISTED THEM HERE IN THE ORDER OF INCREASING COMPLEXITY. THE FIRST ONE IS EASE OF USE HIGH-LEVEL FIT FOR YOUR TRAINING LOOP AND YOU WANT TO DISTRIBUTE YOUR TRAINING IN THAT SETUP. THE SECOND USE CASE IS WHAT WE’VE BEEN TALKING ABOUT IN THIS TALK SO FAR WHERE YOU HAVE A CUSTOM TRAINING LOOP AND YOU WANT TO SCALE OUT YOUR TRAINING. THE THIRD USE CASE IS MAYBE YOU DON’T HAVE A SPECIFIC TRAINING BUT YOU’RE MAYBE A CUSTOM LAYER AND YOU WANT DISTRIBUTION AWARE AND MAYBE YOU’RE EXPERIMENTING WITH A TRAINING ARK TESHURE AND YOU WANT TO CREATE A NEW STRAIGHT? IN THE TALK HERE I’M ONLY GOING TO ATTACK ABOUT THE FIRST TWO CASES. SO LET’S BEGIN. THE FIRST USE CASE WE’LL TALK ABOUT IS IF YOU HAVE A SINGLE MACHINE WITH MULTIPLE GPUS AND YOU WANT TO SCALE UP YOUR TRAINING TO THIS SITUATION. FOR THIS SETUP WE PROVIDE MIRROR STRATEGY. MIRROR STRATEGY IMPLEMENTS SYCHRONOUS TRAINING ACROSS MULTIPLE GPUS ON A SINGLE MACHINE. THE ENTIRE COMPUTON — COMPUTATION ON THE VARIABLES AND ALL THE VARIABLES WOULD BE ON GPU AS WELL AND THEY’LL BE KEPT IN SYNC USING AUTOUSE. LET ME GO THROUGH STEP-BY-STEP TO TALK ABOUT WHAT THE SYCHRONOUS TRAINING LOOKS LIKE. OKAY. THERE WAS SOME GRAY BOXES AROUND THESE WHICH IS NOT REALLY VISIBLE SO LET’S SAY WE HAVE IN OUR EXAMPLE WE HAVE TWO DEVICES OR TWO GPUS DEVICE SERIES IN ONE, AND WE HAVE A VERY SIMPLE LASER — LAYERS, LAYER 1 AND LAYER 2. THE VARIABLES ARE MIRRORED OR REPLICATED OR THESE TWO DEVICES. SO IN OUR FORWARD PASS WE’LL GIVE A DIFFERENT SLICE OF OUR INPUT DATA TO EACH OF THESE DEVICES AND IN THE FORWARD PASS THEY WILL COMPUTES THE LAYERS ON THESE VARIOUS DEVICES. IN THE BACKGROUND PASS EACH DEVICE WILL THEN COMPUTE THE GRADIENTS AGAIN USING THE LOCAL COPY. ONCE EACH DEVICE HAS COMPUTED THE GRADIENTS THEY’LL COMMUNICATE WITH EACH OTHER TO AGGREGATE THESE GRADIENTS AND THIS IS WHERE AUTOUSE WRITE MENTIONED BEFORE. IT’S A CERTAIN ALGORITHMS TO USE GRADIENTS ON DIFFERENT DEVICE AND REDUCE OVERHEAD OF SINKDATIONS BY QUITE A BIT. THERE’S DIFFERENT ALGORITHMS AVAILABLE AND SOME HARDWARE SUCH AS NVIDIA PROVIDES SPECIAL IMPLEMENTATION FOR AUTOUSE FOR THEIR HARDWARE SUCH AS THE NICKLE ALGORITHM. SINCE THESE GRADIENTS HAVE BEEN — THE AGGREGATED RESULT WOULD BE AVAILABLE ON DEVICE SO IN THIS WAY BOTH OF THE DEVICES ARE KEPT IN SYNC AND THE NEXT FORWARD PASS DOESN’T BEGIN UNTIL ALL THE VARIABLES HAVE BEEN UPDATED. SO NOW LET’S TAKE A LOOK AT SOME CODE TO SEE HOW YOU CAN USE MIRROR STRATEGY FOR YOUR TRAINING. THE FIRST ONE IS IF YOU’RE USING THE KERAS HIGH-LEVEL API AND WE’LL GO BACK TO THE CUSTOM TRAINING EXAMPLE RIGHT AFTER THIS. SO THE CODE HERE IS SOME SCALED FORWARD TO TRAIN THE NAME MOBILE NET MODEL BUT THIS TIME USING THE MODEL TO COMPILE AND FIT API IN KERAS. IN ORDER TO CHANGE THIS CODE TO USE MIRROR STRATEGY, ALL YOU NEED TO DO IS ADD THESE TWO LINES OF CODE. THE FIRST THING YOU DO IS CREATE A MIRROR STRATEGY OBJECT AND THE SECOND THING IS YOU MOVE THE REST OF YOUR TRAINING CODE INSIDE THE SCOPE OF THE STRATEGY. PUTTING THINGS INSIDE THE SCOPE LETS IT TAKE CONTROL OF THINGS LIKE VARIABLE CREATION SO ANY VARIABLE THAT YOU CREATE UNDER THE SCOPE OF THE STRATEGY WILL NOW BE MIRRORED VARIABLES.THE S WILL NOW BE MIRRORED VARIABLES. YOU WANT NEED TO MAKE ANY CHANGES IN YOUR CASE BECAUSE WE’VE ALREADY MADE COMPONENTS OF TENSORFLOW DISTRIBUTION AWARE SO IN THIS CASE WE HAVE OPTIMIZED DISTRIBUTION AS WELL AS COMPILE AND FIT. THE CASE WE SAW IS THE SIMPLEST WAY TO SHOW MIRROR STRATEGY AND YOU CAN CUSTOMIZE IT SO LET’S SAY BY DEFAULT IT WILL USE ALL THE GPUS AVAILABLE ON YOUR MACHINE FOR TRAINING BUT IF YOU WANT TO USE ONLY SPECIFIC ONES YOU CAN SPECIFY THEM USING THE DEVICE’S ARGUMENT. YOU CAN ALSO CUSTOMIZE WHAT ALGORITHM YOU WANT TO USE BY USING THE CROSS DEVICE OPS ARGUMENT. SO NOW WE’VE SEEN HOW TO USE MIRROR STRATEGY WHEN USING THE HELLO KERAS MODEL DEVELOPED FOR API. NOW, LET’S GO BACK TO OUR CUSTOM TRAINING LOOP FROM BEFORE AND SEE HOW YOU CAN USE THAT FOR MIRROR STRATEGY. THERE’S A LITTLE BIT MORE CODE IN THIS EXAMPLE BECAUSE WHEN YOU HAVE A CUSTOM TRAINING LOOP YOU HAVE MORE CONTROL OVER WHAT EXACTLY YOU WANT TO DISTRIBUTE AND YOU NEED TO TO DO A LITTLE BIT MORE WORK TO DISTRIBUTE IT AS WELL. SO HERE THIS IS A SCALE OF THE CUSTOM TRAINING WE HAD BEFORE. WE HAD THE MODEL, THE LOSS FUNCTION AND THE OPTIMIZER AND YOU HAVE YOUR TRAINING STEP AND THEN YOU HAVE YOUR AUTOLOOP THAT CREATES OVERUSE DATA AND THE TRAINING STEP. THE FIRST THING YOU NEED TO DO IS THE SAME AS BEFORE. YOU CREATE THE MIRROR STRATEGY OPTIC AND YOU MOVE THE CREATION OF THE MOLTED, THE OPTIMIZER AND SO ON INSIDE THE STRATEGY AND THE PURPOSE IS THE SAME AS BEFORE. BUT AS I MENTIONED YOU NEED TO DO A FEW MORE THINGS SO THIS IS NOT SUFFICIENT IN THIS CASE AND LET’S GO OVER EACH OF THEM ONE BY ONE. THE FIRST THING YOU NEED TO DO IS TO DISTRIBUTE YOUR DATA TYPICALLY AND IF YOU’RE USING TF.DATA SETS AS YOUR INPUT ALL YOU NEED TO DO CALL STRATEGY DEVELOPMENTAL ON YOUR DATASET AND THIS CREATES A DATASET WHICH YOU CAN ITERATE OVER IN A VERY SIMILAR MANNER AS BEFORE. THE SECOND THING YOU NEED TO DO IS TO SCALE YOUR LOSS BY THE GLOBAL BATCH SIZE AND THIS IS VERY IMPORTANT IF YOU — SO THAT THE CONVERGENT CHARACTERISTICS OF YOUR MODEL DO NOT CHANGE AND YOU PROVIDE A HELPER IN THE LIBRARY TO DO SO. AND THE THIRD THING YOU NEED TO DO IS TO SPECIFY WHICH SPECIFIC COMPUTATION YOU WANT TO REPLICATE IN THIS CASE WE WANT TO REPLICATE OUR TRAINING STEP ON EACH REPLICA SO YOU CAN USE STRATEGY.EXPERIMENTAL RUN API TO PROVIDE THE TRAINING FUNCTION AS WELL AS YOUR INPUT AND THERE’S ANOTHER TF FUNCTION BECAUSE WE WANT ALL OF THIS TO RUN AS A TF GRAPH AS WELL SO THOSE ARE ALL THE CODE CHANGES YOU NEED TO MAKE IN ORDER TO TAKE THE CUSTOM LOOP FROM BEFORE AND RUN IN A DISTRIBUTED FUNCTION IN A CONTRIBUTED API. SO LET’S GO BACK AND TAKE A LOOK AT WHAT KIND OF SCALING YOU CAN SEE. SO JUST OUT OF THE BOX ADDING THIS MIRROR STRATEGY TO THE MOBILE EXAMPLE FROM BEFORE WE’RE ABLE TO GET 80% SCALING FROM — WHEN GOING FROM ONE GPU TO 8 GPUS ON A SINGLE MACHINE. IT’S POSSIBLE TO GET EVEN MORE SCALING OUT OF THIS BY DOING SOME MANUAL OPTIMIZATIONS WHICH WE WON’T BE GOING INTO DAY. TO GIVE ANOTHER EXAMPLE OF THE SCALING, WE RAN MULTIGPU FOR RESIDENT THEORY AND IN THIS EXAMPLE WE ALSO USED 16 AND XLA TECHNIQUES THAT TAYLOR TALKED ABOUT BEFORE AND HERE YOU CAN SEE GOING FROM ONE TO 8 GPUS WE’RE ABLE TO GET 85% SCALING AND THIS EXAMPLE WAS USING THE KERAS MODEL THAT FIT API ON THE CUSTOM TRAINING LOOP EXAMPLE. IF YOU’RE INTERESTED YOU CAN LOOK AT THE LINK AT THE BOTTOM AND YOU CAN TRY OUT THIS MODEL YOURSELF. ANOTHER EXAMPLE IS USING THE TRANSFORMER LANGUAGE MODEL TO SHOW THIS IS NOT JUST FOR IMAGES. WE’RE ABLE TO SCALE UP OTHER TYPES OF MODELS AS WELL AND IN THIS CASE WE’RE ABLE TO GET MORE THAN 90% SCALING WHEN RUNNING FROM ONE TO 8 GPUS SO FAR WE’VE BEEN TALKING ABOUT SCALING UP MULTIPLE GPUS ON A SINGLE MACHINE BUT YOU’LL WANT TO SCALE UP TO MULTIPLE OPINIONS MAYBE EVEN MUPPET APPROXIMATELY GPUS OR PERHAPS WITH SEVERAL GPUS WITH THESE USE CASES WE HAVE THE MIRROR STRATEGY. AS THE NAME SUGGESTS IT’S VERY SIMILAR TO THE MIRROR STRATEGY WE’VE BEEN TALKING ABOUT AND IT IMPLEMENTS SYCHRONOUS TRAINING BUT THIS TIME AROUND ALL THE DIFFERENT MACHINE IN YOUR CLUSTER. IN ORDER TO DO THE AUTOUSE IT USES A NEW OP COLLECTIVE CALLED OPS WHICH IS A SINGLE OP IN THE SINGLE TENSORFLOW OP BASED ON A NUMBER OF DIFFERENT FIGURES LIKE THE NETWORK TOPOLOGY AND THE DIFFERENT MACHINES AS WELL AS TENSOR SIZES AND IT COULD ALSO REPRESENT OPTIMIZATION SUCH AS TENSOR FUSION SO, FOR INSTANCE, IF YOU HAVE A LOT OF SMALL TENSORS THAT YOU WANT TO AGGREGATE IT MAY BATCH THEM UP INTO A SINGLE TENSOR IN ORDER TO REDUCE THE LOAD ON THE NETWORK. SO NOW CAN YOU USE IT IT’S SIMILAR TO MIRROR STRATEGY.pYOU LIKE SO, AND THE REST OF YOUR TRAINING ACTUALLY REMAINS THE SAME. ONCE YOU HAVE THE STRATEGY OBJECT YOU CAN PUT THE REST OF YOUR STRATEGY IN SCOPE AND YOU’RE GOOD TO GO. ONE MORE THING YOU NEED TO DO, HOWEVER, IN THE MULT WORKER CASE IS TO GIVE US INFORMATION ABOUT YOUR CLUSTER AND ONE WAY TO DO THAT FOR MULTIWORKER STRATEGY IS USELE TF CONFIG ENVIRONMENT AND YOU MIGHT BE FAMILIAR AND USING ESTIMATOR IN TENSORFLOW 1 SO IT BASICALLY CONSIST OF TWO COMPONENTS AND THE FIRST IS THE CLUSTER WHICH GIVE US INFORMATION ABOUT YOUR ENTIRE CLUSTERS SO HERE WE HAVE 3 DIFFERENT WORKERS IN THE HOST AND PORT AND THE SECOND PIECE IS INFORMATION ABOUT THE SPECIFIC WORKERS OR SAYING THIS IS WORKER 1. ONCE YOU PROVIDE THIS TF CONFIG THE TASK PART WOULD BE DIFFERENT ON EACH WORKER BUT YOUR TRAINING CODE CAN REMAIN THE SAME AND DISTRIBUTION STRATEGY WILL READ THIS TF CONFIG IN VARIABLE AND FIGURE OUT HOW TO COMMUNICATE TO THE DIFFERENT WORKERSSO WE’RE TALKING ABOUT MULTIPLE MACHINES, MULTIPLE GPUS WHAT ABOUT TPUS. YOU’VE PROBABLY HEARD OF TPUS IN THIS CONFERENCE. IT’S HARDWARE BUILT BY GOOGLE TO ACCELERATE MACHINE LEARNING WORKLOADS. YOU CAN USE THEM THROUGH CLOUD TPUS OR YOU COULD EVEN TRY THEM OUT IN COLAB. IT ALLOWS YOU TO BE ABLE TO SCALE UP YOUR TRAINING TO TPUS. IT’S VERY SIMILAR TO MIRROR STRATEGY IN THAT IT IMPLEMENTS SYCHRONOUS TRAINING AND IT USES THE CROSS-COMPLICATION API TO DO THE AUTOUSE ACROSS THE TPU COURSE. YOU CAN USE THIS API TO SCALE YOUR TRAINING TO A SINGLE TPU OR A SLICE OF A POD OR A FULL POD AS WELL, AND IF YOU WANT TO TRIED OUT TT PERHAPS YOU CAN TRY TENSORFLOW NIGHT PIECE OR WAIT FOR THE 2.1 STABLE RELEASE FOR THIS ONE AS WELL. LET’S LOOK AT THE CODE TO SEE HOW YOU CAN USE TP STRATEGY THE FIRST TPU CLUSTER RESOLVER AND GIVE IT THE NAME OR ADDRESS OFF YOUR TPU. ONCE YOU HAVE A CLUSTER IS OVER YOU CAN USE THE EXPERIMENTAL CONNECTIVE CLUSTER API TO CONNECT TO THIS CLUSTER AND ALSO INITIALIZE THE TPU SYSTEM. ONCE YOU’VE DONE THESE 3 THINGS YOU CAN THEN CREATE THE STRATEGY OBJECT, AND PASS THE CLUSTER OR OBJECT TO IT. ONCE YOU HAVE THE STRATEGY OBJECT THE CODE REMAINS THE SAME YOU CREATE THE STRATEGY TO SCOPE AND YOU PUT YOUR TRAINING CODE INSIDE THAT. SO I WON’T BE GOING INTO THAT HERE. SO LOOKING AT THIS CODE YOU CAN SEE DISTRIBUTION STRATEGY MAKES IT REALLY EASY TO SWITCH FROM TRAINING TO MULTIPLE GPUS TO MULTIPLE HAD NOT TO SPECIALIZED HARDWARE LIKE TPUS. SO THAT’S ALL I WAS GOING TO TALK ABOUT. TO RECAP TODAY WE TALKED ABOUT A FEW DIFFERENT THINGS. WE TALKED ABOUT TF DATA TO BUILD SIMPLE AND PERFORMANT INPUT PLANS WE TALKED ABOUT HOW YOU CAN USE TF FUNCTION TO RUN YOUR TRAINING AS A TENSORFLOW GRAPH. WE TALKED ABOUT HOW YOU CAN USE XLA AND MIXED PRISON TO IMPROVE YOUR TRAINING SPEED EVEN FURTHER, AND FINALLY WE TALKED ABOUT THE DISTRIBUTION STRATEGY API TO SCALE OUT YOUR TRAINING TO MORE HARDWAREWE HAVE A NUMBER OF RESOURCES ON OUR WEBSITE, A NUMBER OF GUYS AND TUTORIALS AND WE’D ENCOURAGE YOU TO CHECK THOSE OUT, AND IF YOU HAVE ANY QUESTIONS OR YOU WANT TO REPORT BUGS PLEASE TALK TO EACH OF US ON GITHUB AND WE’LL BE AVAILABLE IN THE EXPO CALL AFTER THE TALK IF YOU HAVE MORE QUESTIONS AND FINALLY THE NOTEBOOK THAT WE’VE BEEN USING IN THE TALK IS LISTED HERE SO IF YOU WANT TO TAKE A PICTURE, YOU CAN CHECK IT OUT LATER. THAT’S ALL, THANK YOU SO MUCH FOR LISTENING. [APPLAUSE] WE HAVE SOME TIME SO WE CAN TAKE SOME QUESTIONS.>>THANKS FOR THE NICE PRESENTATION. SO TWO QUESTIONS BOTH REGARDING INFERENCE. SO YOU MENTIONED HOW WE CAN USE TENSORBOARD FOR PROFILING TO SEE HOW WELL OUR CPUS AND GPUS UTILIZED AND HOW WELL OUR UTILIZERS ARE WORKING.>>IT HAS NO TRAINING INFERENCE IT LOOKS AT OPS AS YOUR EXCUSE IT WILL SHOW UP ALL THE TIME.>>IT KIND OF DEPENDS HOW YOU’RE DOING IT INFERENCE –>>SO YOU’RE NOT USING REALLY ANY OF THE PYTHON APIS IS THAT RIGHT?>>I USE PYTHON APIS –>>YEAH, AND THEN YOU CAN USE TENSORBOARD AS WELL.>>OKAY. REGARDING MIXED PRECISION, WILL IT HELP IN INFERENCE AS WELL AND HOW TO ENABLE THAT >>FCC POTENTIALLY. TYPICALLY FOR INFERENCE THE INTEGER PRECISION FORMATS TEND TO BE MORE IMPORTANT THAN THE FLOATING POINT ONES. IF YOU REALLY CARE ABOUT VERY FAST INFERENCE WITH LOW PRECISION, I WOULD SUGGEST THAT YOU CHECK OUT TF LIGHT CUSS THEY HAVE A LOT OF QUANTITIZATION SUPPORT.>>THANK YOU.>>UH-HUH.>>HELLO, I HAD JUST ONE QUESTION ABOUT HOW TO SAVE YOUR MODEL WHEN YOU ARE TRAINING AND DISTRIBUTE STRATEGY BECAUSE YOUR CODE IS REPLICATED IN ALL THE NOTES. WHAT’S THE –>>YEAH.>>WHAT’S THE OFFICIAL WAY TO DO THIS.>>YES, SO YOU WANT TO SAVE THE ME WOULD?>>YES.>>YOU CAN USE THE STANDARD APIS JUST THE TF.MODEL.SAVE AND WHAT IT’LL DO NOT SAVE THE REPLICATED MODEL IT WILL SAVE A SINGLE COPY OF THE MODEL SO IT’S LIKE ANY OTHER SAVED MODEL.>>OKAY.>>AND YOU CAN RELOAD IT — IF YOU LOAD IT IN ANOTHER STRATEGY THEN, IT WILL GET REPLICATED AT THAT POINT OR YOU CAN USE IT FOR INFERENCE AS ANOTHER STATE MODEL.>>’CAUSE WHAT I EXPERIENCED WAS THAT ALL THE NOTES TRIED TO IS ALIVE MODEL AT THE SAME TIME AND THEN IT WAS AN ISSUE BETWEEN THE CONDITION — WITH THE PLACEMENT WHERE THE MODELS WERE SAVED.>>AND OUT AN ERROR OR –>>YES. IT CRASHED.>>MAYBE WE COULD TALK OFFLIFE. IT’S SUPPOSED TO BORK BUT THERE COULD BE BUGS.>>THANKS. I HAVE A QUICK QUESTION ABOUT MIXED PRECISION AND XLA. WILL THE API AUTOMATICALLY USE THE HARDWARE-RELATED FEATURES LIKE, FOR EXAMPLE, MIXED PRECISION IF YOU HAVE VOLLEY TAR MACHINE IT WILL BE ABLE TO USE VOLTAIRE –>>IT WILL TRY TO USE THE TENSOR CORE AND XLA AND MIXED PRECISION SINNIZES — SORRY, MIXED PRECISION SENATORSES A BUNCH OF TASKS AND XLA WILL GO ALONG AND ACTUALLY TRY — IT WILL ACTUALLY OPTIMIZE A LOT OF THOSE AWAY AND WILL TRY MAKE THE LAYOUT MORE AMENABLE TO MIXED PRECISION IT TENDS TO TALK TO THE HARDWARE AT A VERY LOW-LEVEL WILL >>FROM A USER POINT OF VIEW IT’S TRANSPARENT IF I RUN THE SAME CODE ON THE TPU ON THE OLD TPU AND NEW TPU, IT WILL AUTOMATICALLY TRAIN THE HARDWARE ABILITY?>>YES.>>THANK YOU. I’M REALLY SORRY. I HAVE TO STOP THIS NOW FOR THE LIVE STREAMING. WE CAN TAKE MORE QUESTIONS OUTSIDE THE HALL. SO THANK YOU, PRIA AND TAYLOR. THAT WAS EXCELLENT. [APPLAUSE] ♪>>IT CAN POSSIBLY BE THE LARGEST HOMES IN THE WORLD TODAY. WHEN IT GUEST DECIDES TO SELECT A HOME, ONE OF THE BIGGEST INFLUENCES ARE — THEY WILL TAKE A LOT OF PICTURE OF THE SINGLE ROOM AND FORGET TO TAKE PICTURES OF THE OTHER ROOM.>>THEY ALSO HAVE THE OPTION TO ADD CAPTIONS BUT A LOT OF CASES IT’S TOTALLY OFF.>>WE WERE FACED WITH A CHALLENGE IDENTIFYING WHAT’S ACTUALLY IN THESE PICTURES AND PRESENT THEM PROPERLY ON THE SITE. THAT’S ONE AREA WHERE MACHINE LEARNING CELLS. THE REAL CHALLENGE IS ONE OF SCALE. WE HAD UPWARDS OF HALF A BILLION IMAGES TO GET THROUGH. IT WAS GOING TO TAKE MONTHS TO REALLY GET THROUGH THESE.>>USING TENSORFLOW WE WERE ABLE TO SPEED UP THE PROCESS AND DIFFERENTLY A REASONABLE MODEL. AND TENSORFLOW SERVES THE MODEL RESULTS.>>BEFORE YOU USE THE TOOLS YOU WANT WE DID RESEARCH ON THIS AND FIND RESIDENT 150 WAS ONE OF THE SALIENT PERFORMING MODELS WE USED THAT AS THE BASIC ARCHITECTURE.>>THE TENSORFLOW CROSS APIS AND SERVING AND THE DISTRIBUTED GPUS COMPUTESTIONS.>>THIS ULTIMATELY LED TO A PIPELINE THAT WE CAN DEPLOY TO GO THROUGH HUNDREDS OF IMAGES VERY QUICKLY. ♪>>SO THE END GOAL IS BASICALLY USING THESE CLASSIFICATIONS OF IMAGES IS TO MAKE SURE THEIR FISH SET OF PHOTOS ARE NOT JUST — BUT IT COULD BE OF THE LIVING ROOM AND THE SWIMMING POOL FUTURE APPLICATIONS CAN BE TO DETECT DIFFERENT IMAGES IN HOMES AND IF USERS DECIDE TO SEARCH ON THE WEBSITE FOR SPECIFIC AMENITY TYPES YOU CAN ACTUALLY BUBBLE THAT UP TO THE SURFACE.>>IF YOU LIKE HOW IT OPERATES TODAY, THE REASON IT WAS BUT MACHINE LEARNING BUT MACHINE LEARNING IS ALMOST EVERYWHERE IN THE COUNTRY.>>WE’RE PASSING IN THE HUNDREDS OF THOUSANDS OF MODELS AND WE CAN MAKE BETTER EXPERIENCES FOR OUR GUESTS AND BUILD BUSINESSES AS WELL. ♪>>YOU’VE PROBABLY HEARD A LOT ABOUT AI AND MACHINE LEARNING OVER THE LAST FEW MONTHS AND MAYBE YOU’VE BEEN INSPIRED BY VIDEOS SHOWING WHAT’S POSSIBLE WITH AI AND MACHINE LEARNING. BUT WHAT IS IT REALLY? ONCE YOU GO BEYOND THE HYPE AND GET DOWN TO WRITING CODE, WHAT DOES AI REALLY LOOKS LIKE? WELL, THAT’S WHAT WE’RE GOING TO GO THROUGH IN THIS VIDEO SERIES WHERE WE’LL TEACH YOU WHAT IT’S LIKE TO WRITE CODE FOR MACHINE LEARNING AND HOW IT PROVIDES DIFFERENT, NEW AND EXCITING SCENARIOS THAT WILL HELP YOU WRITE APPLICATIONS THAT BEHAVE MORE LIKE A HUMAN BEING GIVING YOU ARTIFICIAL INTELLIGENCE. I’M LAWRENCE AND I’M GOING TO BE YOUR GUIDE. YOU DON’T NEED TO KNOW A LOT TO GET STARTED AND WE’LL BE USING THE PYTHON LANGUAGE. DON’T WORRY IF YOU’VE NEVER USED IT. IT’S SUPER SIMPLE TO UNDERSTAND AND YOU’LL BE UP AND RUNNING IN NO TIME. SO LET’S START WITH A VERY SIMPLE EXAMPLE. CONSIDER YOU’RE CREATING A GAME OF ROCK PAPER SCISSORS WHEN YOU PLAY THIS WITH A HUMAN IT’S VERY BASIC. ANY CHILD CAN LEARN IT IN A FEW MINUTES. LET’S TAKE A LOOK AT THE MOST BASIC PART OF A GAME THAT THE HUMAN BRAIN IS REALLY GOOD AT. AND THAT’S RECOGNIZING WHAT IT’S ACTUALLY LOOKING AT. SO CONSIDER THESE IMAGES. MOST PEOPLE CAN LOOK AT THEM AND INSTANTLY RECOGNIZE WHICH ONES IS A ROCK, PAPER AND WHICH ONE IS A SCISSORS BUT HOW COULD HOW WOULD YOU PROGRAM A COMPUTER TO RECOGNIZE THEM? THINK ABOUT ALL THE DIVERSITY OF HAND TYPES, SKIN COLOR AND EVEN PEOPLE WHO DO SCISSORS LIKE ME WITH THEIR THUMB STICKING OUT AND THOSE WITH THEIR THUMB IN. IF YOU’VE EVER WRITTEN ANY KIND OF CODE, YOU’LL INSTANTLY RECOGNIZE THIS IS A REALLY, REALLY DIFFICULT TASK AND IT MIGHT TAKE YOU THOUSANDS OR TENNESSEE THOUSANDS OF LINES OF CODE AND WTHAT’S JUST TO PLAY ROCK PAPER SCISSORS WHAT IF THERE’S A DIFFERENT WAY TO TEACH A COMPUTER TO RECOGNIZE WHAT IT SEES. WHAT IF YOU COULD HAVE A COMPUTER LEARN IN THE SAME WAY THAT A HUMAN DOES. THAT’S THE CORE OF MACHINE LEARNING AND THE PATH TO ARTIFICIAL INTELLIGENCE. SO TRADITIONAL PROGRAMMING LOOKS LIKE THIS. YOU HAVE DATA, FOR EXAMPLE, A FEED FROM THE WEBCAM AND YOU HAVE RULES THAT ACT ON THIS DATA. THESE RULES ARE EXPRESSED IN A PROGRAMMING LANGUAGE AND ARE THE BULK OF ANY CODE THAT YOU WRITE ULTIMATELY THESE ROLES WILL GIVE YOU AN ANSWER MAYBE IT SAYS A ROCK, PAPER OR SCISSORS. WHAT IF WE TURN THIS DIAGRAM AROUND AND INSTEAD OF YOU AS THE PROGRAMMER FIGURING OUT THE ROLES YOU INSTEAD GIVE IT ANSWERS WITH THE DATA AND HAVE THE COMPUTER FIGURE OUT WHAT THE RULES ARE. THAT’S MACHINE LEARNING, AND THIS IS WHAT SCISSORS LOOK LIKE AND I COULD HAVE A COMPUTER FIGURE OUT THE PATTERNS THAT MATCH THEM TO EACH OTHER. THEN MY COMPUTER WILL HAVE LEARNED TO RECOGNIZE A ROCK, PAPER AND SCISSORS. THAT’S THE CORE OF BUILDING SOMETHING THAT USES MACHINE LEARNING. YOU GET A SET OF DATA THAT HAS PATTERNS INHERENT IN IT AND YOU GET THOSE PATTERNS IN THERE. WHEN WE LEARN A NEURAL NETWORK THAT’S AS COMPLEX AS ROCK PAPER SCISSORS LET’S USE A MUCH SIMPLE EXAMPLE. TAKE A LOOK AT THOSE NUMBERS THERE’S A RELATIONSHIP BETWEEN THE X AND Y VALUES. CAN YOU SEE IT? IT’S ACTUALLY Y=2 – 1. SO IF YOU SAW THAT HOW DID YOU GET THAT? MAYBE YOU NOTICED THE Y VALUE INCREASES BY 2 WHILE THE AND THEN YOU MAY HAVE SEEN WHEN X WAS ZERO, Y WAS – 1 SO YOU FIGURED WHY=2X – 1 WHICH IS A GOOD GUESS AND YOU TAKE A LOOK AT THE OTHER NUMBERS AND SAW THAT IT WORKED. THAT’S EXACTLY THE PRINCIPLE THAT ALL MACHINE LEARNING WORKS ON. SO LET’S TAKE A LOOK. THIS IS THE ENTIRE CODE THAT YOU CAN USE TO CREATE A MACHINE-LEARNED MODEL THAT FIGURES OUT WHAT MATCHES THESE NUMBERS TO EACH OTHER. DON’T WORRY IF SOME OF IT DOESN’T LOOK VERY FAMILIAR RIGHT NOW, YOU’LL BE ABLE TO PICK TPU IN NO TIME. THIS FIRST LINE DEFINES THE MODEL ITSELF. A MODEL IS A TRAINED NEURAL NETWORK AND HERE WE HAVE THE SIMPLEST NEURAL NETWORK WHICH IN THIS CASE IS A SINGLE LAYER INDICATED BY THE KERAS.LAYERS.DENSE CODE AND THAT LAYER HAS A SINGLE NEURON IN IT INDICATED BY UNITS=1. WE ALSO FEED A SINGLE VALUE INTO TH NEURAL NETWORK WHICH IS THE X VALUE AND WE’LL HAVE THE NEURAL NETWORK PREDICT WHAT THE Y WOULD BE FOR THAT X SO THAT’S WHY WE SAY THE INPUT SHAPE IS 1 VALUE. WHEN YOU COMPILE THE MODEL THERE’S TWO FUNCTIONS THE LOSS AND THE IDENTIFIER. HOW MACHINE LEARNING WORKS IS THAT THE MODEL WILL MAKE A GUESS ABOUT THE RELATIONSHIP BETWEEN THE NUMBERS. FOR EXAMPLE, IT MIGHT GUESS THAT Y=5 + 5 AND WHEN TRAINING IT WILL CALCULATE HOW GOOD OR HOW BAD THAT GUESS IS USING THE LOSS FUNCTION AND THEN IT WILL USE THE OPTIMIZER FUNCTION TO GENERATE ANOTHER GUESS. THE LOGIC IS THAT THE COMBINATION OF THESE TWO FUNCTIONS WILL SLOWLY GET US CLOSER AND CLOSER TO THE CORRECT FORMULA, AND IN THIS CASE IT WILL GO THROUGH THAT LOOP 500 TIMES MAKING A GUESS, CALCULATING HOW ACCURATE THAT GUESS IS AND THEN USING THE OPTIMIZER TO ENHANCE THAT GUESS AND SO ON. THE DATA ITSELF IS SET UP AS AN ARRAY AS XS AND YS ON OUR PROCESSOR. ♪ ♪>>IT FIGURES OUT WHAT MATCHES NIECE NUMBERS TO EACH OTHER. DON’T WORRY IF SOME OF IT DOESN’T LOOK VERY FAMILIAR RIGHT NOW. YOU’LL BE ABLE TO PICK THAT UP IN NO TIME. THIS FIRST LINE DEFINES THE MODEL ITSELF. A MODEL IS A TRAINED NEURAL NETWORK AND HERE WE HAVE THE SIMPLEST POSSIBLE NEURAL NETWORK WHICH IN THIS CASE IS A SINGLE LAYER INDICATED BY THE KERAS.LAYERS.DENSE CODE AND THAT LAYER HAS A SINGLE NEURON IN IT INDICATED BY UNITS=1 AND WE ALSO SEND A VALUE TO THE NEURAL NETWORK WHICH IS THE X VALUE WHICH WOULD PREDICT WHAT WOULD Y WOULD BE FOR THE X. THAT’S WHY WE SAY THE INPUT SHAPE IS 1 VALUE. WHEN YOU COMPILE THE MODEL THERE ARE TWO FUNCTIONS. THE LOSS AND THE OPTIMIZER. THESE ARE THE KEYS TO MACHINE LEARNING. HOW MACHINE LEARNING WORKS IS THAT THE MODEL WILL MAKE A GUESS ABOUT THE RELATIONSHIP BETWEEN THE NUMBERS. FOR EXAMPLE, IT MIGHT GUESS THAT Y=5 + 5 AND WHEN TRAINING IT WILL THEN CALCULATE HOW GOOD OR HOW BAD THAT GUESS IS USING THE LOSS FUNCTION AND THEN IT WILL USE THE OPTIMIZER FUNCTION TO GENERATE ANOTHER GUESS THE LOGIC IS THAT THE COMBINATION OF THESE TWO FUNCTIONS WILL SLOWLY GET US CLOSER AND CLOSER TO THE CORRECT FORMULA, AND IN THIS CASE IT WILL GO THROUGH THAT LOOP 500 TIMES MAKING A GUESS, CALCULATING HOW ACCURATE THAT GUESS IS AND THEN USING THE OPTIMIZER TO ENHANCE THAT GUESS AND SO ON. THE DATA ITSELF IS SET UP AS AN ARRAY OF XS AND WHYS IN OUR PROCESS MIXING THEM EACH OTHER IS A FIT IN THE MODEL. WE LITERALLY SAY 50XS TO THE YS AND TRY THIS 500 TIMES. WHEN IT’S DONE, WE’LL HAVE A TRAINED MODEL SO NOW YOU CAN TRY TO PREDICT THE Y VALUE FOR A GIVEN X. WHAT DO YOU THINK WOULD HAPPEN IF YOU TRY THIS LINE OF CODE, PREDICT THE Y WHEN X=10. YOU MIGHT THINK THAT THE ANSWER IS 19; RIGHT? BUT IT ISN’T. IT’S ACTUALLY SOMETHING LIKE 18.9998. IT’S CLOSE TO 19 BUT IT’S NOT QUITE THERE. WHY DO YOU THINK THAT WOULD BE? WELL, THE COMPUTER WAS TRAINED TO MATCH ONLY 6 PAIRS OF NUMBERS. IT LOOKS LIKE A STRAIGHT LINE RELATIONSHIP BETWEEN THEM FOR THOSE 6 BUT IT MAY NOT BE A STRAIGHT LINE FOR VALUES OUTSIDE OF THOSE 6. THERE’S A VERY HIGH PROBABILITY THAT IT’S A STRAIGHT LINE BUT WE CAN’T BE CERTAIN. AND THIS PROBABILITY IS BUILT INTO THE PREDICTION SO IT’S TELLING US A VALUE VERY CLOSE TO 19 INSTEAD OF EXACTLY 9. TRY THE CODE OUT USING THE LYMPH IN THE DESCRIPTION BELOW THIS VIDEO TO SEE IF FOR YOURSELF. THIS IS SOMETHING YOU’LL SEE A LOT MORE OF IN MACHINE LEARNING. IN THE NEXT VIDEO IN THIS SERIES WE’LL TAKE WHAT YOU’VE LEARNED AND APPLY THAT TO A MORE INTERESTING PROBLEM COMPUTAVISION AND SEEING HOW YOU CAN TEACH A COMPUTER TO SEE THINGS USING EXACTLY THE SAME METHODOLOGY AS YOU USED HERE. WE’LL SEE YOU IN THAT VIDEO AND DON’T FORGET TO HIT THAT SUBSCRIBE BUTTON. THANK YOU. ♪>>HI, EVERYONE AND WELCOME TO EPISODE 2 OF TENSORFLOW 0 TO HERO. IN THE LAST EPISODE YOU LEARNED ABOUT MACHINE LEARNING AND HOW IT WORKS. YOU SAW A SIMPLE EXAMPLE OF LEARNING MATCHED NUMBERS TOGETHER AND HOW USING A PYTHON CODE A COMPUTER COULD LEARN THROUGH TRIAL AND ERROR WHAT THE RELATIONSHIP BETWEEN THE NUMBERS WAS. IN THIS EPISODE YOU’RE GOING TO TAKE IT A LITTLE FURTHER BY TEACHING A COMPUTER HOW TO SEE AND RECOGNIZE DIFFERENT OBJECTS. FOR EXAMPLE, LOOK AT THOSE PICTURES. HOW MANY SHOES DO YOU SEE YOU MIGHT SAY 2, BUT HOW DO YOU KNOW THEY ARE SHOES. IMAGINE IF SOMEBODY HAD NEVER SEEN SHOES BEFORE HOW WOULD YOU TELL THEM THAT DESPITE THE GREAT DIFFERENCE BETWEEN THE HIGH HEEL AND THE SPORTS SHOE, THEY’RE STILL BOTH SHOES? MAYBE THEY WOULD THINK IF IT’S RED IT’S A SHOE BECAUSE ALL THEY’VE SEEN ARE THESE TWO AND THEY’RE BOTH RED BUT, OF COURSE, THAT’S NOT THAT SIMPLE. BUT HOW DO YOU KNOW THESE TWO ARE SHOES BECAUSE IN YOUR LIFE YOU’VE SEEN LOTS OF SHOES AND YOU’VE LENDERS TO UNDERSTAND WHAT MAKES A SHOE A SHOE. SO IT FOLLOWS LOGICALLY IF WE SHOW A COMPUTER LOTS OF SHOES IT WILL BE ABLE TO RECOGNIZE WHAT A SHOE IS. AND THAT’S WHERE THE DATASET CALLED FASHION MS IS USEFUL. IT HAS 70,000 IMAGES IN 10 DIFFERENT CATEGORIES SO THE 7,000 EXAMPLES OF EACH INQUIRY INCLUDING SHOES. HOPEFULLY, SEEING 7,000 SHOES IS ENOUGH FOR A COMPUTER TO LEARN WHAT A SHOW LOOKS LIKE. THE IMAGES ARE ONLY 28X28 PIXELS SO THEY’RE PRETTY SMALL, AND THE LESS DATA USED THE FASTER IT IS FOR A COMPUTER TO PROCESS IT. THAT BEING SET, THEY STILL LEAD TO RECOGNIZABLE ITEMS OF CLOTHING. IN THIS CASE YOU CAN STILL SEE THAT IT’S A SHOE. IN THE NEXT FEW MINUTES I’LL SHOW YOU THE CODE THAT WILL TEACH YOU HOW TO TRAIN A COMPUTER TO RECOGNIZE ITEMS OF CLOTHING BASED ON THIS TRAINING DATA. THE TYPE OF CODE YOU WRITE IS ALMOST IDENTICAL TO WHAT YOU DID IN THE LAST VIDEO. THAT’S PART OF THE POWER OF TENSORFLOW THAT U.S. LOOK YOU TO DESIGN NEURAL NETWORKS FOR A VARIETY OF TASKS WITH A CONSISTENT API. WE’LL START BY LOADING THE DATA. THE DATASET IS BUILT INTO TENSORFLOW SO IT’S EASY TO LOAD IT WITH CODE LIKE THIS. THE TRAINING IMAGES IS A SET OF 60,000 IMAGES LIKE OUR ANKLE BOOT HERE. THE OTHER 10,000 ARE A TEST SET THAT WE CAN USE TO CHECK TO SEE HOW WELL OUR NEURAL NETWORK PERFORMANCE. WE’LL SEE THEM LATER. THE LABEL IS A NUMBER INDICATING THE CLASS OF THAT TYPE OF CLOTHING SO IN THIS CASE THE NO. 9 INDICATES AN ANKLE BOOT. WHY DO YOU THINK IT WOULD BE A NUMBER AND NOT JUST THE TEXT ANKLE BOOT? THERE’S TWO MAIN REASONS, FIRST COMPUTERS DEAL BETTER WITH NUMBERS BUT PERHAPS MORE IMPORTANTLY THERE’S THE ISSUE OF BIAS. IF WE LABEL IT AS ANKLE BOOT WE’RE ALREADY SHOWING A BIAS TOWARDS THE ENGLISH LANGUAGE. SO BY USING A NUMBER, YOU CAN POINT TO A TEXT DESCRIPTION IN ANY LANGUAGE AS SHOWN HERE. CAN YOU GUESS ALL THE LANGUAGES THAT WE USED HERE? WHEN LOOKING AT A NEURAL NETWORK DESIGN IT’S ALWAYS GOOD TO EXPLORE THE INPUT VALUES AND THE OUTPUT VALUES FIRST. HERE WE CAN SEE THAT OUR NEURAL NETWORK IS A LITTLE MORE COMPLEX THAN THE ONE IN THE FIRST EPISODE. OUR FIRST LAYER HAS THE INPUT OF SHAPE 28X28 WHICH IF YOU REMEMBER WAS THE SIZE OF OUR IMAGE. OUR LAST LAYER IS 10 WHICH IF YOU REMEMBER IS THE NUMBER OF DIFFERENT ITEMS OF CLOTHING REPRESENTED IN OUR DATASET. SO OUR NEURAL NETWORK WILL KIND OF ACT LIKE A FILTER WHICH TAKES IN A 20X IT WILL SET OF PIXELS AND OUTPUTS 1 OF 10 VALUES. SO WHAT ABOUT THIS NUMBER, 128, WHAT DOES THAT DO? WELL, THINK OF IT LIKE THIS WE’RE GOING TO HAVE A 128 FUNCTIONS, EACH ONE OF WHICH HAS PARAMETERS INSIDE OF IT. LET’S CALL THESE F0 THROUGH F127. WHAT WE WANT IS THAT WHEN THE PIXELS OF THE SHOE GET FED INTO THEM, ONE BY ONE, THAT THE COMBINATION OF ALL OF THESE FUNCTIONS WILL OUTPUT THE CORRECT VALUE IN THIS CASE 9. IN ORDER TO DO THAT THE COMPUTER WILL NEED TO FIGURE OUT THE PARAMETERS IN EACH OF THOSE FUNCTIONS TO GET THE RESULT AND IT WILL EXTEND TO ALL THE OTHER ITEMS OF CLOTHING AND THE DATASET. THE LOGIC IS ONCE IT HAS DONE THIS, THEN IT SHOULD BE ABLE TO RECOGNIZE ITEMS OF CLOTHING. SO IF YOU REMEMBER FROM THE LAST VIDEO THERE’S THE OPTIMIZER FUNCTION AND THE LOSS FUNCTION. THE NEURAL NETWORK WILL BE INITIALIZED WITH RANDOM VALUES. THE LOSS FUNCTION WILL THEN MEASURE HOW GOOD OR HOW BAD THE RESULTS WERE AND THEN WITH THE OPTIMIZER IT WILLATE OPERATE NEW PARAMETERS FOR THE FUNCTIONS TO SEE IF IT CAN DO BETTER. YOU PROBABLY ALSO WONDERED ABOUT THESE AND THEY’RE CALLED ACTIVATION FUNCTIONS. THE FIRST ONE IS ON THE LAYER OF 128 FUNCTIONS AND IT’S CALLED RELU OR RECTIFIED LINEAR UNIT AND WHAT IT REALLY DOES AS SIMPLE AS RETURNING A VALUE THAT’S GREATER THAN ZERO SO IF THAT FUNCTIONED ZERO OR LESS AS OUTPUT, IT JUST GETS FILTERED OUT, AND SOFTMAX HAS THE EFFECT OF PICKING THE BIGGEST NUMBER IN A SET. THE OUTPUT LAYER IN THIS NEURAL NETWORK HAS 10 ITEMS IN IT REPRESENTING THE PROBABILITY THAT WE’RE LOOKING AT THAT SPECIFIC ITEM OF CLOTHING. SO IN THIS CASE IT HAS A HIGH PROBABILITY THAT IT’S ITEM 9 WHICH IS OUR ANKLE BOOT SO INSTEAD OF SEARCHING THROUGH TO FIND THE LARGEST, WHAT SOFTMAX DOES IT SETS IT TO ONE AND THE REST TO ZERO SO ALL WE HAVE TO DO IS FIND THE 1. TRAINING IS THEN VERY SIMPLE. WE FIT THE TRAINING IMAGES TO THE TRAINING LABELS. THIS TIME WE’LL TRY IT FOR JUST 5 EPOCHS. REMEMBER EARLIER WE HAD 10,000 IMAGES AND LABELS THAT WE DIDN’T TRAIN WITH THESE ARE IMAGES THAT THE MODEL HASN’T PREVIOUSLY SEEN SO WE CAN USE THEM TO TEST HOW WELL OUR MODEL PERFORMS. WE CAN DO THAT TEST BYPASSING THEM TO THE EVALUATE METHOD LIKE THIS. AND THEN FINALLY WE CAN GET PREDICTIONS BACK FOR NEW IMAGES BY CALLING MODEL.PREDICT LIKE THIS. AND THAT’S ALL IT TAKES TO TEACH A COMPUTER HOW TO SEE AND RECOGNIZE IMAGES. YOU CAN TRY THIS OUT FOR YOURSELF IN THE NOTEBOOK THAT I’VE LINKED IN THE DESCRIPTION BELOW. HAVING GONE THROUGH THIS, YOU’VE PROBABLY SEEN ONE DRAW-BACK AND THAT’S THE FACT THAT THE IMAGES ARE ALWAYS 28X28 GRAYSCALE WITH THE ITEM OF CLOTHING CENTERED. SO WHAT IF IT’S JUST THE NORMAL PHOTOGRAPH AND WE WANT TO RECOGNITION ITS CONTENTS AND YOU DON’T HAVE THE LUXURY OF IT BEING THE ONLY THING IN THE PICTURE AS WELL AS BEING CENTERED? THAT’S WHY THE PROCESS OF SPOTTING FEATURES BECOMES USEFUL AND THE ALPHA CONVULATIONAL NETWORKS IS YOUR FRIEND. YOU’LL LEARN ALL ABOUT THAT IN THE NEXT VIDEO SO DON’T FORGET TO HIT THAT SUBSCRIBE BUTTON AND I’LL SEE YOU THERE. ♪>>HI, AND WELCOME TO EPISODE 3 OF 0 TO HERO WITH TENSORFLOW. YOU SAW HOW TO DO BASIC COMPUTER VISION USING A DEEP NEURAL NETWORK THAT MATCHED THE PIXELS OF AN IMAGE TO A LABEL. SO AN IMAGE LIKE THIS WAS MATCHED TO A NUMERIC LABEL THAT REPRESENTED IT LIKE THIS. COLUMBUS THERE WAS A LIMITATION TO THAT. THE IMAGE YOU WERE LOOKING AT HAD TO HAVE THE SUBJECT CENTERED IN IT AND IT HAD TO BE THE ONLY THING IN THE IMAGE SO THE CODE YOU WROTE WOULD WORK FOR THAT SHOE BUT WHAT ABUT THESE? IT WOULDN’T BE ABLE TO IDENTIFY ALL OF THEM BECAUSE IT’S NOT TRAINED TO DO SO. WITH THAT WE HAVE TO USE SOMETHING CALLED A CONFUSIONAL NEURAL NETWORK WHICH WORKS A LITTLE DIFFERENTLY THAN WHAT YOU’VE JUST SEEN. THE IDEA BEHIND A CONVOLUTIONAL NEURAL NETWORK IS THAT YOU FILTER THE IMAGES BEFORE TRAINING THE DEEP NEURAL NETWORK. AFTER FILTERING THE IMAGES FEATURES WITHIN THE IMAGES COULD THEN COME TO THE FOREFRONT AND YOU WOULD THEN SPOT THOSE FEATURES TO IDENTIFY SOMETHING. A FILTER IS SIMPLY A SET OF MULT FLIERS SO, FOR EXAMPLE, IN THIS CASE IF YOU’RE LOOKING AT A PARTICULAR PIXEL THAT HAS THE VALUE 192 AND THE FILTER IS THE VALUES IN THE RED BOX, THEN YOU MULTIPLY 192X4.5 AND EACH OF ITS NEIGHBORS BY THE RESPECTIVE FILTER VALUE SO ITS NEIGHBOR ABOVE AND TO THE LEFT IS ZERO SO YOU MULTIPLY THAT BY – 1. IT’S UPPER NEIGHBOR IS 64 SO YOU MULTIPLY THAT BY ZERO AND SO ON. SUM UP THE RESULT AND YOU GET THE NEW VALUE FOR THE PIXEL. NOW, THIS MIGHT SEEM A LITTLE ODD BUT CHECK OUT THE RESULTS FOR SOME FILTERS LIKE THIS ONE. THAT WHEN MULTIPLIED OVER THE CONTENTS OF THE IMAGE IT REMOVES ALMOST EVERYTHING EXCEPT THE VERTICAL LINES AND THIS ONE IT REMOVES ALMOST EVERYTHING EXCEPT THE HORIZONTAL LINES. THIS CAN THEN BE COMBINED WITH SOMETHING CALLED POOLING WHICH GROUPS UP THE PIXELS IN THE IMAGE AND FILTERS THEM DOWN TO A SUBSET SO, FOR EXAMPLE, MAX POOLING 2X2 WILL GROUP IMAGE INTO SETS OF 2X2 PIXELS AND SIMPLY PICK THE LARGEST. THE IMAGE WILL BE REDUCED TO A QUARTER OF ITS ORIGINAL SIZE BUT THE FEATURES CAN STILL BE MAINTAINED, SO THE PREVIOUS IMAGE AFTER BEING FILTERED AND THEN MAX POOLED COULD LOOK LIKE THIS. THE IMAGE ON THE RIGHT IS ONE QUARTER THE SIDES OF THE OF THE ONE ON THE LEFT BUT THE VERTICAL LINE FEATURES WERE MAINTAINED AND INSTEAD THEY WERE ENHANCED. SO WHERE DID THESE FILTERS COME FROM AND THAT’S THE MAGIC OF A CONVERSIONAL NEURAL NETWORK THEY’RE ACTUALLY LEARNED THEY’RE JUST PARAMETERS LIKE THIS IN THE NEURONS OF A NEURAL NETWORK THAT WE SAW IN THE LAST VIDEO SO AS OUR IMAGE IS FED INTO THE CONFUSIONAL LAYER, A NUMBER OF RANDOMLY INITIALIZED FILTERS WILL PASS OVER THE IMAGE. THE RESULTS OF THESE ARE FED INTO THE NEXT LAYER AND MATCHING AS PERFORMED BY THE NEURAL NETWORK. AND OVER TIME THE FILTERS THAT GAVE US THE IMAGE OUTPUTS THAT GIVE THE BEST MATCHES WILL BE LEARNED AND THE PROCESS IS CALLED FEATURE EXTRACTION. HERE’S AN EXAMPLE OF HOW A CONFUSIONAL FILTER LAYER CAN HELP A COMPUTER VISUALIZE THINGS. YOU CAN SEE ACROSS THE TOP ROW HERE THAT YOU ACTUALLY HAVE A SHOE BUT IT HAS BEEN FILTERED DOWN TO THE SOUL AND THE SILHOUETTE OF A SHOE BY FILTERS THAT LEARNED WHAT A FILTER LOOKS LIKE. YOU’LL RUN THIS CODE FOR YOURSELF IN JUST A FEW MINUTES. NOW LET’S TAKE A LOOK AT THE CODE TO BUILT A CONFUSIONAL NEURAL NETWORK LIKE THIS. SO THIS CODE IS VERY SIMILAR TO WHAT YOU USED EARLIER WHERE YOU HAVE A FLATTENED INPUT THAT’S FED INTO A DENSE LAYER THAT IN TURN IS FED INTO THE FINAL DENSE LAYER THAT IS OUR OUTPUT. THE ONLY DIFFERENCE HERE IS THAT I HAVEN’T SPECIFIED THE INPUT SHAPE. THAT’S BECAUSE I’LL PUT A CONFUSIONAL LAYER ON TOP OF IT LIKE THIS. THIS LAYER TAKES THE INPUT SO WE SPECIFY THE INPUT SHAPE, AND WE’RE TELLING IT TO GENERATE 64 FILTERS WITH THIS PARAMETER. THAT IS IT WILL GENERATE 64 FILTERS AND MULTIPLY EACH OF THEM ACROSS THE IMAGE. THEN EACH EPOCH IT WILL FIGURE OUT WHICH FILTERS GAVE THE BEST SIGNALS TO HELP MATCH THE IMAGES TO THE LABELS IN MUCH THE SAME WAY IT LEARNED WHICH PARAMETERS WORKED BEST IN THE DENSE LAYER. THE MAX POOLING TO COMPRESS THE POOLING AND ENHANCE THE FEATURES LOOKS LIKE THIS. AND WE CAN STACK CONFUSIONAL LAYERS ON TOP OF EACH OTHER TO REALLY BREAK DOWN THE IMAGE AND TRY TO LEARN FROM VERY ABSTRACT FEATURES LIKE THIS. WITH THIS METHODOLOGY YOUR NETWORK STARTS TO LEARN BASED ON INSTEAD OF THE RAW PATTERNS OF PIXELS. TWO SLEEVES IT’S A SHIRT, TWO SHORT SLEEVES IT’S A T SHIRT SOUL AND LACES IT’S A SHOE THAT TYPE OF THING WE’RE STILL LOOKING AT THE SIMPLE IMAGES OF FASHION AT THE MOMENT BUT THE PRINCIPLES WILL EXTEND INTO MORE COMPLEX IMAGES AND YOU’LL SEE THAT IN THE NEXT VIDEO. BUT BEFORE GOING THERE, TRY OUT THE NOTEBOOK TO SEE CONVOLUTIONS, AND BEFORE WE GET TO THE NEXT VIDEO DON’T FORGET TO HIT THAT SUBSCRIBE BUTTON. THANK YOU. ♪>>HI, EVERYBODY AND WELCOME TO THE 4TH AND FINAL VIDEO IN THIS SERIES OF 0 TO HERO WITH TENSORFLOW. I’M LAWRENCE AND TODAY WE’RE GOING TO LOOK BACK AT THE VERY FIRST PROBLEM THAT WE SPOKE ABOUT. AND THEN WE’LL SEE HOW WE CAN BUILD A MACHINE-LEARNED MODEL TO SOLVE IT. REMEMBER THIS? ALL THE WAY BACK IN EPISODE 1 WHERE WE SHOWED A SCENARIO OF ROCK, PAPER AND SCISSORS AND DISCUSSED HOW DIFFICULT IT MIGHT BE TO CREATE AN APPLICATION THAT RECOGNIZES HANDS OF DIFFERENT SHAPES, SIZES, ETHNICITIES, DECORATIONS AND MORE. WE DISCUSSED HOW DIFFICULT IT WOULD BE TO WRITE CODE TO DETECT AND CLASSIFY THESE EVEN FOR SOMETHING AS SIMPLE AS A ROCK PPER OR SCISSORS BUT SINCE THEN, YOU’VE LOOKED INTO MACHINE LEARNING AND YOU’VE SEEN HOW TO BUILD NEURAL NETWORKS FIRST TO DETECT PATTERNS IN RAW PIXELS TO CLASSIFY THEM AND THEN TO DETECT FEATURES USING CONVOLUTIONS TO HAVE A CONFUSIONAL NEURAL NETWORK TRAINED TO SPOT THE PARTICULAR FEATURES THAT MAKE UP AN ITEM LIKE THE SOULS OF A SHOE. LET’S PUT ULAPPEAR TOGETHER AND IN THIS VIDEO WE’LL SEE HOW TO CREATE A NEURAL NETWORK THAT IS TRAINED ON DATA OF ROCK, PAPER AND SCISSORS TO DETECT AND SPOT THEM. WE’LL START WITH THE DATA. THERE’S A DATASET HERE THAT HAS SEVERAL HUNDRED IMAGES OF ROCK, PAPER AND SCISSORS POSES. WE’LL TRAIN A NEURAL NETWORK WITH THIS DATA. SO FIRST OF ALL, WE HAVE TO DOWNLOAD THE STIPULATE FILES CONTAINING THE DATA. THE CODE TO DO THAT IS HERE. ONE FILE HAS A TRAINING SET AND THE OTHER HAS A TESTING AND VALIDATION SET. IN PYTHON YOU CAN UNZIP A FILE WITH THE ZIP FILE, LIKE, AND WE UNZIPPED A FILE LIBRARY LIKE THIS. THIS CREATES FOLDERS WITH SUBFOLDERS OF EACH OF OUR CATEGORIES. WHEN TRAINING IN TENSORFLOW USING AN IMAGE DATA GENERATOR IT WILL AUTOMATICALLY LABEL THE IMAGES BASED ON THE NAME OF THEIR PARENT DIRECTORY, SO I DON’T NEED TO CREATE LABELS FOR THE IMAGES. IT’S A REALLY NICE SHORTCUT. SO TO ACHIEVE THAT WITH THIS CODE IT HAS A IMAGE DATA GENERATOR FROM THE TRAINING TO THE DISTRICTRY THAT THEY WERE DOWNLOADED TO. WE COULD SET UP SOMETHING CALLED A TRAINING GENERATOR WHICH AS ITS NAME SUGGESTS CREATES TRAINING DATA FROM THAT. WE CAN DO EXACTLY THE SAME FOR THE TEST SET WITH THIS CODE. LATER WHEN YOU SEE THE MODEL.FIT YOU’LL SEE WE PASS IT IN FROM THE TRAINING TO THE VALIDATION PARAMETERS. NOW, LET’S LOOK AT OUR NEURAL NETWORK DEFINITION. THIS IS VERY, VERY LIKE WHAT YOU SAW IN THIS VIDEO JUST LIKE WITH MORE LAYERS. ONE REASON THE IMAGES ARE MORE COMPLICATED THAN THE GRAYSCALE CLOTHING YOU SAW PREVIOUSLY AND THE OTHER IS THAT THEY’RE BIGGER. YOU CAN SEE THAT OUR INPUT IS NOW 150X150 OUR IMAGES ARE DEFINITELY BIGGER THAN THEY WERE BEFORE AND OUR OUTPUT IS A LAYER OF 3 NEURONS. WELL, WHY WOULD THAT BE? BECAUSE THERE ARE 3 CLASSES, ROCK, PAPER AND SCISSORS. BETWEEN THESE THE CODE IS VERY SIMILAR TO WHAT YOU SAW PREVIOUSLY, JUST MORE OF IT. SO WE HAVE FOUR LAYERS OF CONVOLUTIONS EACH WITH MAX POOLING BEFORE FEEDING INTO A DENSE LAYER THE DROPOUT IS A LITTLE TRICK TO IMPROVE THE EFFICIENCY OF A NEURAL NETWORK BY THROWING AWAY SOME OF THE NEURONS. WE’LL COMPILE A NEURAL NETWORK AS BEFORE WITH THIS CODE AND THEN WE CAN FIT THE DATA WITH THE MODEL.FIT CALL. NOTE THAT WE DON’T HAVE LABELS. THAT’S BECAUSE WE’RE USING THE GENERATOR. IT’S INFERRING THE LABELS FROM THE PARENT DIRECTORIES IN BOTH THE TRAINING AND THE VALIDATION DATASETS. WHEN YOU RUN THIS YOU’LL PROBABLY GET ACCURACY OF ABOUT 100% OF THE TRAINING DATA QUITE QUICKLY WITH THE VALIDATION DATA GETTING ABOUT 70% ACCURACY. THIS IS SOMETHING CALLED OVERFITTING WHICH HAPPENS WHEN THE MODEL GET REALLY GOOD AT SPOTTING WHAT IT HAS SEEN BEFORE BUT IT’S NOT SO GREAT AT GENERALIZING. THINK ABOUT IT THIS WAY SO, FOR EXAMPLE, IF ALL YOUR LIFE THE ONLY SHOES YOU HAD EVER SEEN WERE HIKING BOOTS, YOU PROBABLY WOULDN’T RECOGNIZE HIGH-HEELED SHOES. YOU’D BE OVERFITTING YOURSELF. THERE ARE A NUMBER OF METHODS TO AVOID THIS AND ONE OF THEM IS CALLED IMAGE AUGMENTATION AND I’VE PUT THE CODE FOR THIS INTO THE NOTEBOOK FOR YOU TO TRY YOURSELF TO SEE HOW IT HELPS AVOID OVERFITTING. ONCE YOUR MODEL IS TRAINED YOU CAN THEN CALL MODEL.PREDICT TO SEE HOW WELL IT SPOTS ROCK PAPER OR SCISSORS. THIS CODE WILL TAKE AN IMAGE, REFORMAT IT TO 150X150 WHICH THE MODEL IS TRAINED FOR AND IT WILL THEN RETURN A PREDICTION, AND HERE’S A FEW EXAMPLES THAT I RAN SO THAT YOU CAN SEE ALL THE WAY IT’S ACTUALLY PREDICTING QUITE WELL BUT THE BEST THING TO DO IS TO TRY IT FOR YOURSELF. I’VE PUT A LINK FOR THE NOTEBOOK IN THE DESCRIPTION BELOW AND YOU CAN USE THIS CODE TO TRAIN A NEURAL NETWORK TO TRAIN ROCK PAPER SCISSORS IMAGES. THAT’S IT FOR THIS SHORT SET OF VIDEOS. I HOPE IT WAS USEFUL FOR THE NEW PARADIGM THAT IS MACHINE LEARNING AND WITH THE FEW EXAMPLES HOW YOU CAN GET YOURSELF ON THE PATH TO BECOME AN ARTIFICIAL INTELLIGENCE ENGINEER. IF YOU’VE ANY QUESTIONS, PLEASE LEAVE THEM IN THE COMMENTS BELOW AND DON’T FORGET TO SUBSCRIBE FOR MORE GREAT CONTENT. THANK YOU.>>HI, EVERYBODY. I’M LAWRENCE MARONEY FROM THE TENSORFLOW FROM GOOGLE. IT’S ALL ABOUT COMMUNITY AND IN THE SHOW TENSORFLOW MEETS WE WANT TO TALK ABOUT WHAT YOU’RE DOING WITH YOUR TENSORFLOW.>>PEOPLE WILL BE ABLE TO AUTHOR MODELS USING THE KERAS INTERFACE.>>AND THEN DEPLOY THEM RIGHT INTO A BROWSER.>>THAT WILL BE HUGE.>>AND SO YOU DO IT?>>SO IF YOU HAVE A PROJECT OR RESEARCH OR ANYTHING YOU’D LIKE TO TALK ABOUT JUST DROP US A LINE AND MAYBE YOU’LL SEE YOURSELF ON THE SHOW. ♪>>HI, EVERYBODY WELCOME TO TENSORFLOW MEETS I’M LAWRENCE MALONEY, AND ARUN I KNOW YOU’VE BEEN DOING GREAT STUFF WITH PROBLEMLISTIC MODELING FOR THOSE WHO DON’T UNDERSTAND TELL US ABOUT IT.>>GOOD MORNING AND THANK YOU FOR HAVING ME HERE. AND YES, SO PROBLEMLISTIC MODELING AND PROBLEMLISTIC TRAINING IS SOMETHING WE’VE BEEN USING FOR SEVERAL YEARS NOW.>>UH-HUH.>>IT’S MOSTLY FOR MODELING SYSTEMS THAT HAVE A COMBINATION OF VERY COMPLEX PHENOMENON. COUPLED WINNING THAT WE CAN’T MEASURE PRECISELY.>>OKAY.>>AND TO JUST GIVE YOU A SIMPLE EXAMPLE IF I WERE TO ASK YOU TO PREDICT WHERE THE STONE WOODLAND IF YOU THREW IT.>>OKAY.>>THEN ANY HIGH SCHOOL STUDENT WOULD TELL YOU THAT THEY CAN CALCULATE IT PRECISELY BASED ON HOW FAST YOU THREW IT AND AT WHAT ANGLE YOU THREW THE STONE.>>OKAY.>>IF I WERE TO ADD A LITTLE BIT OF UNCERTAINTY TO IT SAYING I DON’T KNOW EXACTLY WHAT ANGLE YOU THREW THE STONE AT OR WHAT VELOCITY –>>LIKE WIND SHEAR –>>AND STUFF LIKE THAT, THEN ALL OF A SUDDEN, YOUR PREDICTIONS ARE NO LONGER AS PRECISE. NOW, IN A SIMPLE SYSTEM LIKE THAT YOU CAN ALREADY SEE THINGS STARTING TO GET COMPLEX. IMAGINE A COMPLICATED SYSTEM IN THE WORLD THINGS CAN GET MUCH MORE COMPLEX IF YOU DON’T PREDICT SOMETHING PRECISELY SOME>>SO YOU’RE WORKING ON A LOT OF COMPLEX SYSTEMS LIKE THIS CAN YOU SHARE SOME EXAMPLES?>>EXAMPLE. WE WORK ON A LOT OF COMPLEX SYSTEMS, SAY, DESIGNING GUEST — OR TRYING TO PREDICT WHAT A VERY LARGE SCALE SYSTEM LIKE AN OFFSHORE OIL PLATFORM WOULD DO. WE’RE TALKING ABOUT00 OR THOUSANDS OF VARIABLES INTERACTING WITH EACH OTHER AND MOST OF THE TIME YOU CAN PREDICT OR YOU CAN MEASURE MAYBE A FEW HUNDREDS OR A FEW 10TH OF THOSE. HOW DO YOU HAVE COMPLEX SYSTEMS AND HAVE ACTIONABLE MEANING FOR MODELS WITHOUT KNOWING ALL THE SYSTEMS.>>I SEE, OKAY. IT SOUNDS COMPLEX. SO WHAT IS YOUR APPROACH TO THIS? HOW DO YOU GET STARTED?>>SO WE GET STARTED WITH STARTING WITH THE DOMAIN SO WE UNDERSTAND THE DOMAIN YOU MAY BE A MECHANICAL ENGINEER OR ARREST AEROSPACE ENGINEER YOU START WITH THE UNDERSTANDING OF THE DOMAIN AND MARRY THAT WITH TRADITIONAL MACHINE LEARNING TECHNIQUES THAT’S BEEN GOING ON FOR SEVERAL DECADES THAT GIVES YOU A VERY GOOD UNDERSTANDING OF HOW TO PREDICT THINGS PRECISELY AND THAT’S WHAT WE CALL KNOWN KNOWNS SO WE CAN PREDICT THE KNOWN THINGS.>>KNOWN KNOWNS.>>CORE AREA WE CAN START FROM AND THEN WE ADD A LAYER OF PROBABLISICS ON TOP WHAT ARE THE THINGS THAT WE CANNOT MEASURE PRECISELY OR MEASURE AT ALL.>>OKAY.>>AND THAT’S WHAT THE PROBLEMISTIC MODELING COMES IN AND THAT’S WHAT I CALLED KNOWNS UNKNOWNS.>>OKAY. AN EXAMPLE WOULD SAY IF I’M TRYING TO PREDICT HOW A CRACK IS GOING TO PROP INDICATE IN A PARTICULAR — THEN I NEED TO KNOW WHAT IS THE TEMPERATURE OF THAT PARTICULAR CONFERENCE, FOR EXAMPLE. I MEASURED WITHIN PLUS OR MINUS 10 DEGREES BUT I DON’T KNOW WHAT THAT VARIATION IN TEMPERATURE IS GOING TO DO TO MY CROWD PROP INDICATING SO THAT’S WHAT I WOULD CALL KNOWN AND UNKNOWNS.>>OKAY.>>ONCE I KNOW WHAT ARE THE UNKNOWNS THAT I’M NOT ENTIRELY SURE ABOUT, I CAN GO SAY, OKAY, THIS IS WHAT THE IMPACT OF THAT AND SOMETHING REAL. THERE’S ANOTHER LEVEL OF COMPLEXITY WHERE THINGS THAT I DON’T KNOW I DON’T KNOW.>>IT GOT.>>THAT’S WHAT I WOULD CALL UNKNOWNS, AND IT’S A MUTHFUL AN EXAMPLE OF THAT WOULD BE SAY YOU HAVE DESIGNED A SYSTEM AND YOU HAVE PUT IT OUT IN THE REAL WORLD, YOU KNOW, SOME OF THE SYSTEM BUT YOU’RE NOT ENTIRELY SURE EVERYTHING THAT’S GOING TO AFFECT THE SYSTEM AND THAT IS THAT OTHER EVERYTHING THAT’S WHAT WE WOULD CALL UNKNOWN UNKNOWNS, AND MOST OF THE TIME IN THE REAL WORLD YOU COULD PREDICT SOMETHING UP TO 90 TO 95% THE LAST 5% IS WHAT SURPRISES US AND IN SYSTEMS THAT ARE SAFETY CRITICAL TO CLEAN UP THE STRUCTURE OF THE WORLD, YOU CAN’T NECESSARILY HAVE EVEN A 1% CHANCE OF SOMETHING GOING DOWN, RIGHT, SO, FOR EXAMPLE, IF POWER GOES DOWN, YOU NEED TO BE ABLE TO BRING THE BACKUP REALLY QUICKLY AND THOSE ARE THE THINGS THAT ARE KNOWN AND UNKNOWN.>>GOT IT STARTING FROM KNOWN KNOWNS AND GOING TO THE KNOWABLE UNKNOWNS –>>EXACTLY.>>I SEE SO YOU’VE GONE FOR KNOWN KNOWNS TO KNOWABLE UNKNOWNS –>>UNKNOWNS UNKNOWNS. WHEN YOU’RE TRYING TO MODEL SYSTEMS THAT ARE HIGHLY COMPLEX AND ARE EXTREMELY CRITICAL YOU NEED TO BE ABLE TO PREDICT THINGS AT ALL OF THOSE LEVELS AND EVEN IF YOU’RE NOT ABLE UNKNOWNS. WHEN AN EVENT HAPPENS HOW WOULD YOU RESPOND TO THAT THAT’S WHERE THE KNOWN UNKNOWNS COMES.>>BRINGING THIS INTO JUST DEVELOPING THESE THINGS YOU USE TENSORFLOW PROBABILITY?>>YES.>>WE STARTED WITH TENSORFLOW AND COMBINED THAT WITH TENSORFLOW PROBABILITY QUITE A BIT.>>SO COULD YOU TELL US A LITTLE BIT ABOUT HOW YOU USE ALL OF THAT.>>ABSOLUTELY.>>WE STARTED TENSORFLOW FOR DEEP LEARNING PRECISELY AND WE GOT INVOLVED IN THE PROBABILITY TEAM WHAT WE REALIZED THEY WERE BRINGING EXTREMELY DEEP RESEARCH CONCEPTS FROM THE PROBLEMISTICS WORLD INTO A PRODUCTION WORLD THAT’S GENERALLY NOT MONCKS CORNER AND WE WERE ABLE TO MIX THE DEEP LEARNING COMMUNITY WITH THE PROBLEMISTICS COMMUNITY WITHIN OUR OWN TEAMS AND RUNNING A REASONABLY LARGE DATA SCIENCE TEAM WHAT YOU HAVE TO DO IS MIX TEAMS THAT ARE NOT NECESSARILY IT ALL TOGETHER SAME LANGUAGE.>>OKAY.>>AND TENSORFLOW ALLOWS US TO DO THAT VERY EFFECTIVELY BECAUSE A DEEP LEARNING EXPERT WHO DOESN’T UNDERSTAND PROBLEMISICS VERY WELL CAN TALK TO A EXPERT WHO DOESN’T UNDERSTAND DEEP PLANNING IN THE SAME LANGUAGE.>>NICE, SO HAVING THAT FRAMEWORK THAT THEY WOULD WORK TOGETHER IS VERY POWERFUL.>>ABSOLUTELY IT HELPS SCALE OUR TEAMS AS WELL AS OUR DEPLOYMENTS VERY QUICKLY.>>WOW, SO A LOT OF COMPLEX STUFF THAT YOU’VE BEEN WORKING ON. THERE MUST BE SOMEHOW YOU GOT STARTED TO FIGURE OUT THIS — HOW DID YOU LEARN ALL THIS?>>ABSOLUTELY. I’M NOT A TRAINED DATA SCIENTISTS FOR TRAINING. I GOT INTO DATA SCIENCE BY ACCIDENT. I WAS A AEROSPACE ENGINEER HAVING TO MIX THESE TOGETHER.>>THAT’S A VERY COMMON STORY.>>ONE OF THE THINGS HAPPENED ME TRYING TO MIX PRACTICAL ASPECTS FOR THE PRACTICAL ASPECTS THE BOOK I LOVE IS DOING VISION DATA ANALYSIS AND THAT GIVES — AT LEAST IT GAVE ME QUITE A BIT OF UNDERSTANDING HOW THESE ARE APPLIED IN THE REAL WORLD BUT AT THE SAME TIME THINKING ABOUT PROBABILITY REQUIRES PEOPLE TOMORROW MORNING ABOUT SOLVING PROBLEMS IN A VERY, VERY FUNDAMENTALLY DIFFERENT WAY BECAUSE WE’RE GOOD AT BEING GOOD AT HOW DO WE — BUT IF THE SAME INPUTS GIVE YOU MULTIPLE OUTCOMES THAT’S A VERY DIFFERENT PARADIGM TO THINK ABOUT. SO A SET OF BOOKS THAT HELPED ME WERE FROM EDIE JAMES WHICH WERE A MORE PHILOSOPHICAL UNDERSTANDING PROBABILITIES. I WOULD URGE FOLKS TO DABBLE IN BOTH THE PRACTICAL ASPECTS AS WELL AS SOME BIT OF THE FILL SORT OFFICS TOGETHER AND IF YOU LOOK AT THE RECENT BLOGS FROM THE TENSORFLOW TEAM AS WELL AS THE BROADER COMMUNITY IN DOING PROBLEMLISTIC DEEP LEARNING THERE’S A LOT OF FANTASTIC PLOS OUT THERE THAT WILL HELP PEOPLE GET STARTED AS WELL.>>AND I’D SAY ONE THING I KNOW YOU’VE WRITTEN A COUPLE BLOGS AND THERE’S ANOTHER IN A WAY WHERE YOU’VE BEEN GOING INTO A LITTLE BIT MORE DETAIL THAN WHAT WE GOT THROUGH.>>WE WALKED THROUGH THE 3 BLOGS BECAUSE WE WANTED TO WORK IN KNOWN KNOWNS, UNKNOWN KNOWNS –>>AND THE ONE YOU’RE STILL WORKING ON IS THE UNKNOWNS.>>AND IT’S GETTING PUBLISHED IN THE NEXT MONTH OR SO.>>ALL THAT IS ON BLOGS.TENSORFLOW..ORG.>>YES.>>THANK YOU EVERYBODY FOR WATCHING TENSORFLOW MEETS IF YOU HAVE ANY QUESTIONS FOR ME OR ARUN JUST PLEASE LEAVE THE COMMENTS BELOW AND WE’LL PUT LINKS TO EVERYTHING THAT WE PUT TOGETHER SO YOU CAN CHECK THEM OUT FOR YOURSELF.>>THANKS FOR HAVING ME.>>>>SO THIS IS OUR MOBILE SOUND SCAPE EXHIBIT WHERE WE GO FROM THERE.>>SOUND WAVES ARE VIBRINGINGS IN THE AIR.>>IT’S VAST AND STRANGE ENVIRONMENT. AS LONG AS YOU HAVE CURIOSITY YOU KNOW THERE’S LIFE. WHEN I LOOK OUT TO THE OCEAN I GET THAT SAME FEELING AS I DO WHEN I LOOK UP INTO THE STARS. ♪>>HEARING MY PARENTS PLAY GUITAR THAT WAS PROBABLY MY FIRST MEMORY. AS A KID I WOULD RUN THROUGH THE HALLWAYS AND I WOULD HEAR THEM PRACTICING TRADITIONAL MEXICAN MUSIC –>>HERE WE ARE AT THE PLAZA THE PLAZA PRINCIPAL.>>MY MOM DIDN’T KNOW SPANISH AND MY FATHER DIDN’T KNOW ENGLISH BUT THEY WERE ABLE TO COMMUNICATE AT LEAST WITH MUSIC AND I CAME ALONG. [LAUGH]>>DANNY, HE WAS A BUSY ONE. VERY INQUISITIVE WHEN HE WAS LITTLE BUT HIS CURIOSITY SOMETIMES GOT HIM IN TROUBLE.>>I WAS NOT A GOOD STUDENT ACTUALLY. I DIDN’T HAVE ANY EXPECTATIONS I WASN’T LOOKING AT UNIVERSITIES OR ANYTHING LIKE THAT AFTER GRADUATING I THOUGHT I MIGHT AS WELL GO TO COMMUNITY COLLEGE. I STARTED AT CABRUNN AND I DIDN’T KNOW WHAT ENGINEERING WAS AS FAR AS I WAS CONCERNED THAT WAS A MECHANIC OR SOMETHING LIKE THAT.pBUT I STARTED TAKING SOME CLASSES AND IT WAS DIFFERENT >>YOU KNOW, THERE’S LOTS OF TYPES OF STUDENTS. THE MAJORITY OF THE STUDENTS THEY WANT THE EQUATION SO THAT THEY COULD CALCULATE A NUMBER AND THEN THERE ARE STUDENTS WHO ACTUALLY TRY TO UNDERSTAND. THAT’S DANIEL.>>I WOULD ALWAYS QUESTION THINGS AND I WOULD ALWAYS WONDER WHY LIGHT WORKS AND THE WAVES CRASH THE WAY THEY DO AND THE SOUND WAVES ARE JUST VIBRATIONS IN THE AIR AND HOW WE REPRESENT MATHEMATICALLY AND MY EYES OPENED TO THE WORLD AND THAT’S WHEN IT CLICKED THAT I KNEW I WANTED TO SEE ENGINEERING. CABRERA COLLEGE HELD A SYMPOSIUM WHERE THEY CONNECTED STUDENTS TO THE INSTITUTIONS. IT WAS REAL INTERESTING WHAT DANIELLE WAS DOING.>>DANIEL HAD A MUSICAL BACKGROUND THAT’S AN IMPORTANT PART ON WORKING ON AUDIO YOU HAVE TO HAVE AN UNDERSTANDING OF THE WAY SOUND WORKS.>>LIGHT MAY REACH A COUPLE HUNDRED METERS IN THE OCEAN. SOUND CAN TRAVEL THOUSANDS OF MILES. IT’S NOT A SURPRISE THAT MANY LIFE FORMS USE SOUND FOR COMMUNICATING. BLUE-GRAY WHALE IS THE LARGEST NUMBER TO HAVE EVER LIVED ON EARL I EARTH INCLUDING BIGGER THAN A DINOSAUR.>>IT’S INCREDIBLE TO HEAR.>>THE SCIENTISTS WANTED TO FIGURE OUT A WAY TO LISTEN IN AND COUNT HOW MANY BLUE WHALES THERE WERE.>>SO THE PROCESS THAT WE’RE USING INVOLVES RECORDING AUDIO FROM THE OCEAN 24 HOURS A DAY, 7 DAYS A WEEK. THAT’S A LOT OF INFORMATION TOO MUCH INFORMATION FOR A HUMAN TO TRY TO LOOK AT AND LISTEN TO.>>WE WERE ABLE TO TURN THOSE SOUNDS TO SPECTRUMS WHICH ARE JUST IMAGES OR MAPPINGS OF WHAT SOUND LOOKS LIKE AND FED THOSE IMAGES INTO TENSORFLOW. THIS MACHINE LEARNING TOOL.>>MACHINE LEARNING IS GETTING THE COMPUTER TO BE ABLE TO PICK UP PATTERNS.>>IT ALLOWS US TO TAKE A MAMMOTH PILE OF DATA AND DISTILL IT INTO SOMETHING MEANINGFUL SO THAT WE CAN DO THE ANALYSIS.>>WE WANT TO UNDERSTAND HOW AND WHEN THEY’RE USING ALL THESE CALLS ‘CAUSE THAT CAN HELP US UNDERSTAND THEIR ECOLOGY.>>WE CAN ANSWER A LOT OF QUESTIONS ABOUT THE WAY WE ARE AFFECTING THE MARINE ENVIRONMENT AND HOW WE CAN HELP CONSERVATIVE IT.>>DANIEL WAS KEY IN DEVELOPING THE SOFTWARE TO AUTOMATICALLY FIND THE CALLS FOR US.>>DANIEL STARTED WITH THE ART ARREST TISRY OF SOUND AND MOVING TO THE SCIENCE OF SOUND. ULTIMATELY BOTH OF EXPERIENCE — THEY’RE FROM THE SAME SOURCE. THAT IS SOUND ENGINEERING VARYING IN FREQUENCY THROUGH TIME. THAT’S WHAT MAKES MUSIC AND THAT IS THE SIGNAL WE ANALYZE.>>THE WHALED ARE COMMUNICATING SOMETHING TO EACH OTHER. I DON’T KNOW WHAT IT IS BUT IT MADE ME THINK ABOUT HOW MUSIC IS A WAY OF COMMUNICATING. JUST LIKE GOING BACK TO MY PARENTS AND HOW THEY MET EACH OTHER. THEY WERE DOING THE SAME THING. ♪ WHEN I WAS A KID I DIDN’T THINK I WAS GOING TO BE A SCIENTIST OR AN ENGINEER ESPECIALLY AS I GREW UP I DIDN’T THINK I WAS CAPABLE OF DOING IT. SCHOOL WAS HARD FOR ME AND IT STILL HARD FOR ME. I DON’T KNOW IF I FOUND MY PASSION HI FIND MYSELF MORE INTERESTED IN A LOT OF DIFFERENT THINGS BUT KNOWING THAT I’LL NEVER STOP LEARNING MAKES ME FEEL PRETTY LUCKY. ♪>>WE’VE GOT A GREAT TALK, WE HAVE JOANA CARRASQUEIRA AND NICOLE PANG, PROJECT MANAGER FOR EDUCATION IN TensorFlow. THEY’RE GOING TO TALK TO YOU ABOUT HOW YOU CAN GET INVOLVED IN THE COMMUNITY. [ APPLAUSE ]>>THANK YOU. THANK YOU SO MUCH, JOCELYN. FIRST OF ALL, WELCOME EVERYBODY. IT’S AN ABSOLUTE PLEASURE TO BE HERE WITH YOU TODAY. AS JOCELYN MENTIONED, I’M A PROGRAM MANAGER FOR TENSORFLOW. I’M JOINED BY MY COLLEAGUE NICOLE PANG.>>Nicole: I’M A PROJECT MANAGER FOR TensorFlow.>>Joana: WE’RE GOING TO TALK ABOUT THE TensorFlow COMMUNITY AND THE MANY EXCITING WAYS BY WHICH YOU CAN GET INVOLVED IN THE WORK THAT WE DO. SO, LET ME START BY SAYING THANK YOU. THANK YOU TO YOU. THANK YOU TO THE COMMUNITY FOR ALL THE HARD WORK THAT YOU’VE DONE. SINCE WE’VE OPEN SOURCED TensorFlow IN 2015, WE’VE RECEIVED SO MANY CONTRIBUTIONS AND SO MUCH SUPPORT FROM THE COMMUNITY THAT REALLY, THE PROJECT WHERE IT LEADS TODAY IS DUE TO ALL YOUR EFFORTS AND HARD WORK. THANK YOU FOR THAT. JUST ON CORE TensorFlow ALONE, WE’VE RECEIVED MORE THAN 6,000 COMMITS FROM OVER 2,000 CONTRIBUTORS. THIS IS SO IMPRESSIVE. BUT NOT JUST ONLY THIS. ON STACK OVERFLOW, WE HAVE RECEIVED MORE THAN 50,000 QUESTIONS AND WE HAVE ONBOARDED MORE THAN 120 MACHINE LEARNING EXPERTS THROUGH OUR GOOGLE DEVELOPER EXPERT PROGRAM. AND WE HAVE ESTABLISHED 50 USER GROUPS ALL AROUND THE WORLD. WE’VE ALSO HAD 25 GUEST POSTS ON OUR TensorFlow BLOG, WHICH IS FANTASTIC. AND OUR COMMUNITY ONLY CONTINUES TO GROW. HERE IS A NAPSHOT WHERE YOU CAN SEE THAT THE NUMBER OF COMMITS FROM FOUR YEARS AGO HAS BEEN RAPIDLY GROWING. AND THERE’S SO MUCH SUPPORT AND EXCITEMENT FROM THE COMMUNITY. WE TRULY COULDN’T HAVE GOTTEN THIS FAR IF IT WASN’T FOR YOU, FOR THE CONTRIBUTORS, FOR ALL THE WORK THAT YOU DO. SO, THANK YOU SO MUCH FOR THAT.>>Nicole: AND IT’S NOT JUST THE CONTRIBUTIONS YOU SEE IN THE FEEDBACK WE GET FROM OUR COMMUNITY ON GitHub AND STACK OVERFLOW, BUT, OF COURSE, AS YOU ALL KNOW, TensorFlow IS A GLOBAL — HAS A GLOBAL WORLDWIDE COMMUNITY NE. WE SEE A LOT OF LOVE FOR TensorFlow ON OTHER AVENUES. YOU HAVE HAD HEARD ABOUT TF 2.0. BUT, IT’S ONE INSTANCE WHERE OUR GLOBAL COMMUNITY RESPONDS REALLY POSITIVELY AND WE SEE SO MANY USES OF THAT. AND TODAY WE’LL TOUCH ON THESE CASES AND, OF COURSE, HOW YOU CAN GET INVOLVED IN OUR COMMUNITY. SO, BRIEFLY WHAT WE’LL TALK ABOUT TODAY — WE WANT TO TELL YOU HOW YOU CAN LEARN TensorFlow, HOW YOU CAN GET STARTED IN YOUR OWN JOURNEY OF USING TensorFlow, WHETHER YOU’RE MORE IN THE BEGINNING STAGES, OR YOU’RE A RELEVANCED USER. WE WANT TO SHOWCASE TO YOU OUR GLOBAL COMMUNITY, RUN THROUGH SOME AMAZING USE CASES, REALLY TELL YOU WHAT WE’VE SEEN PEOPLE USE TensorFlow FOR, AND HOPEFULLY THAT CAN BE VERY INSPIRATIONAL FOR ALL OF US IN THE COMMUNITY. AND, OF COURSE, WHILE YOU’RE HERE TODAY YOU WANT TO KNOW HOW TO GET INVOLVED. WE’LL WALK YOU THROUGH NOT JUST THE WAYS YOU MIGHT FIRST THINK OF, WHICH MIGHT BE CONTRIBUTING CODE, BECAUSE TensorFlow IS OPEN SOURCE, BUT ALSO A LOT OF COMMUNITY GROUPS, A LOT OF SPECIAL INTEREST GROUPS. AND THOSE ARE ALL OVER THE WORLD. BOTH FOR EVERYONE HERE IN THIS ROOM, AND EVERYONE WATCHING ONLINE, THERE’S MANY RESOURCES AND WE’RE SO EXCITED TO SHARE WITH YOU.>>SO, WE TRULY, AS YOU CAN SEE, WE TRULY HAVE A VIBRANT GLOBAL COMMUNITY THAT CONTINUES TO GROW, BECAUSE THERE’S SO MUCH THAT YOU CAN DO, SO MUCH THAT WE CAN ALL CONTRIBUTE TO TensorFlow. AND LET’S HAVE A LOOK AT WHERE OUR COMMUNITY IS BASED AND WHAT ARE THEY DOING RIGHT NOW. SO, THE TensorFlow USER GROUPS, THEY ARE A WONDERFUL WAY IN GETTING INVOLVED WITH TensorFlow, EITHER ONLINE OR FACE-TO-FACE. YOU CAN MEET WITH OTHER LIKE-MINDED CONTRIBUTORS AND DEVELOPERS, REALLY TO ANSWER QUESTIONS, TO SOLVE PROBLEMS, CHALLENGES, AND BUILDING THOSE USE CASES ON REALLY HOW YOU CAN IMPLEMENT TensorFlow ACROSS DIFFERENT INDUSTRIES. SO, JUST AN EXAMPLE, ONE OF OUR USER GROUPS IN KOREA — THAT ONE IS THE BIGGEST WE HAVE IN THE WORLD. WE HAVE ENGAGED MORE THAN 46,000 MEMBERS. IT IS VERY IMPRESSIVE. AND IN CHINA ALONE IT, IT HAS TE MOST USER GROUPS, ACROSS 15 DIFFERENT CITIES. IT’S REALLY IMPRESSIVE HOW THE COMMUNITY IS GROWING SO FAST ALL OVER THE WORLD. AND ONE OF THE KEY MESSAGES THAT NICOLE AND I WOULD LIKE YOU TO RETAIN FROM OUR PRESENTATION TODAY IS THAT IF YOU DON’T HAVE A USER GROUP WHERE YOU’RE BASED, OR IN YOUR REGION, FEEL FREE TO START ONE. SHARE YOUR EXPERIENCES. CONNECT WITH OTHER LIKE-MINDED DEVELOPERS. AND START TALKING ABOUT TensorFlow. WE ARE HERE TO SUPPORT YOU THROUGHOUT THIS PROCESS AND YOUR JOURNEY. SO FEEL FREE TO REACH OUT TO US. WE’RE VERY HAPPY TO GUIDE YOU THROUGH THE PROCESS. AND LIKE I MENTIONED, IF YOU WOULD LIKE TO START YOUR USER GROUP, HERE ARE SOME OF THE RESOURCES THAT YOU CAN HAVE A LOOK ONLINE IF YOU ARE INTERESTED IN STARTING YOUR OWN GROUP. WE ARE SHARING OUR ALIAS. SO YOU CAN REALLY GET TO KNOW THE TEAM AND HOW YOU CAN START CREATING YOUR USER GROUP.>>Nicole: YEAH. SO IN THE SPIRIT OF HONORING OUR GLOBAL COMMUNITY, WE WANT TO BRIEFLY TOUCH ON WHAT THE TensorFlow TEAM HAS BEEN GOING WORLDWIDE. LIKE JOANA JUST SAID, WE HAVE SO MANY USER GROUPS. YOU REALLY CAN SEE THAT THEY ARE GLOBAL. AS YOU HEARD THIS MORNING, THE TensorFlow TEAM WAS EXCITED AND LUCKY TO BE ABLE TO GO TO MANY CITIES AND MEET MANY OF THESE USERS AND THE COMPANIES, AND THE STARTUPS THAT ARE USING TensorFlow IN SO MANY DIFFERENT CITIES. WE’RE EXCITED YOU’RE HERE TODAY, ONE OF OUR STOPS IN SANTA CLARA TODAY. AND WE’RE REALLY EXCITED TO, AGAIN, BE ABLE TO SEE THE USE CASES. AND WE’D LOVE TO SHARE BRIEFLY SOME OF THE USE CASES WITH YOU. SO, FIRST OFF, WHEN WE LOOK AT ASIA AND ASIA PACIFIC, THERE’S A REALLY BIG, VIBRANT COMMUNITY THERE. AS JOANA JUST SAID, A LOT OF PEOPLE IN KOREA, INDIA AND CHINA ARE USING TensorFlow WITH TWO AMAZING APPLICATIONS. IN CHINA, TensorFlow IS NOT JUST ACTIVE ON APPLICATIONS, BUT ALSO THE COMMUNITY IS ACTIVE ON OUR OFFICIAL TensorFlow WE CHAT CHANNEL. THIS CHANNEL SHOWCASES A LOT OF USE CASES ON MOBILE. YOU CAN SEE THIS ONE EXAMPLE OF A VIDEO PLATFORM CALLED ITE WITH IMAGE SEGMENTATION ON MOBILE DEVICES. THEY’RE DOING REALLY AWESOME WORK, AND NOT JUST DOING AWESOME WORK, BUT ALSO SHARING WITH ALL OF THE COMMUNITY ON THE WE CHAT BLOG. WE’RE REALLY GLAD THAT WE’RE PARTNERING WITH THEM AND GLAD TO SEE THESE USE CASES COME UP.>>AND NICOLE AND I WERE REALLY FORTUNATE THAT WE’RE ABLE TO JOIN THE ROAD SHOWS AND CONNECT WITH THE LOCAL COMMUNITIES WORLDWIDE. SO FOR EXAMPLE, AT THE ROAD SHOW IN LATIN AMERICA, WE CONNECTED WITH A STARTUP IN CHILE AND THEY ARE TRYING TO DETECT SUPERNOVAS AND GALAXIES THROUGH THE USES OF AGILE PROCESSES AND MACHINE LEARNING. AND THIS WAS REALLY COOL. AND THEY USE CONVENTIONAL NEURAL NETWORKS TO CLASSIFY ASTRONOMICAL OBJECTS CONTAINED IN A STREAM OF ABOUT 200,000 IMAGES PER DAY. THE WORK THAT THEY’RE DOING IS SO IMPRESSIVE AND IT’S ABSOLUTELY WORTH SHARING WITH THE REST OF THE COMMUNITY. ANOTHER EXAMPLE, IN EUROPE, WE CONNECTED WITH IM, WHICH IS A LIBRARY OF PHOTOS THAT USES TensorFlow FOR OBJECT CLASSIFICATION. AND THEIR ALGORITHM SCORES PHOTOS BASED ON THEIR STATIC QUALITY, BUT ALSO ON THE RELEVANCE TO YOUR BRAND’S VISUAL IDENTITY. AND THEN EVERY PHOTO IS AUTOMATICALLY TAGGED WITH KEY WORDS JUST TO MAKE SURE THAT THE ENTIRE LIBRARY IS SEARCHABLE. IT’S REALLY IMPRESSIVE. AND THEY USED TensorFlow LITE TO MAKE IT MORE ACCESSIBLE FOR USERS TO USE IM. LASTLY, IN AFRICA, WE MET WITH MANY EXCITING STARTUPS TRYING TO FIND SOLUTIONS FOR PROBLEMS AT A GLOBAL SCALE THAT WERE RELEVANT TO THE REGION. WE WOULD LIKE TO HIGHLIGHT THE GREAT WORK OF SOMEONE WHO LEV LEVERAGES THE POWER OF MERGE AND SPECTRUM ANALYSIS TO REALLY TURN ANY SMARTPHONE INTO A POWERFUL, NONINVASIVE SCREENING TOOL FOR ASTHMA AND OTHER PULMONARY DISEASES. SO THEY USE CONVOLUTION NEURAL NETWORK FOR MODELING SPECTROGRAMS, AND THEN THE MODELS ARE FROZEN AND CONVERTED INTO TensorFlow LITE. AND THE CONVERTED MODEL IS THEN DEPLOYED TO A MOBILE DEVICE TO PERFORM INTERFERENCE. SO, THESE WERE SOME OF THE CASES THAT WE CONNECTED WITH DURING THE ROAD SHOWS, AND IT WAS BRILLIANT TO SEE ALL THESE VERY INNOVATIVE WAYS THAT THE COMMUNITY IS USING AND BUILDING AROUND TensorFlow. SO, THESE WERE JUST A FEW PICTURES OF OUR ROAD SHOWS WHERE WE TRULY ENGAGED WITH THE COMMUNITY AND IT’S PALPABLE. IT’S VERY TANGIBLE THE EXCITEMENT THAT WE SEE NOT ONLY FROM CONTRIBUTORS, BUT ALSO FROM USERS OF TensorFlow. IT’S FANTASTIC TO SEE HOW MANY OF THESE STARTUPS AND OTHER COMPANIES ARE TRULY IMPACTING AND CHANGING THE WORLD. AND THIS IS ALL USING TensorFlow. SO, THANK YOU FOR THAT.>>Nicole: YEAH. SO LIKE WE SAID IN THE BEGINNING, YOU KNOW, WE WANTED TO DO A BRIEF OVERVIEW OF SOME — JUST A VERY SMALL SAMPLE OF SOME OF THE AWESOME USE CASES OF TensorFlow. BUT THEN REALLY DIG INTO WHAT IS AVAILABLE FOR YOU, RIGHT? SO ONE OF THE FIRST PILLARS THAT WE’LL TALK ABOUT IS EDUCATION. NOW, WHY IS EDUCATION IMPORTANT FOR US AT TensorFlow AND ALSO WE HOPE IT’S IMPORTANT FOR YOU IN THE COMMUNITY? WELL, TensorFlow IS, OF COURSE, AS YOU ARE VERY HEARTILY KNOWING, IT’S OPEN-SOURCE. BUT ALSO ANOTHER ASPECT OF THAT OPEN SOURCE NATURE IS THAT WE WANT TO MAKE SURE LEARNING RESOURCES ARE AVAILABLE TO EVERYONE IN THE WORLD. AND WE REALLY VALUE NOT MAKING JUST THE PRODUCTS BETTER FOR LEARNERS — FOR INSTANCE, TF 2.0, ITS EASY DEBUGGING AND THE USABILITY OF KARATES IS DESIGNED FOR THAT BETTER EXPERIENCE FOR LEARNERS. SO NOT JUST THE PRODUCT, BUT ALSO THE EDUCATIONAL RESOURCES. I’D LOVE TO GO INTO THAT IN MORE DETAIL. THIS MORNING YOU HEARD ABOUT OUR LAUNCH OF THE NEW LEARN HUB ON TensorFlow.ORG. THIS IS A GREAT TOOL, BECAUSE WE HEARD PEOPLE’S FEEDBACK THAT THEY WOULD LIKE MORE CURATED RESOURCES ON TensorFlow.ORG. THEY WOULD LIKE MORE PATHS OF LEARNING FROM, YOU KNOW, WHATEVER LEVEL OF MACHINE LEARNING AND DEEP LEARNING KNOWLEDGE YOU HAVE INTO MORE ADVANCED APPLICATIONS. SO WE HEARD YOU AND WE NOW RESPONDED WITH THIS NEW RESOURCE OF LEARN ML. IT’S NOT JUST A COMPILATION OF CURATED RESOURCES, BUT IT’S ALSO GUIDED PATHS, WHETHER YOU’RE A BEGINNER ON TensorFlow, WHETHER YOU’RE MORE ADVANCED, WHICH RESOURCES AND TUTORIALS, GUIDES MIGHT BE HELPFUL. IF YOU ARE INTERESTED IN TFJS ON THE BROWSER, WE HAVE A VERY DETAILED, VERY NICELY ORGANIZED LEARNING RESOURCE. WE HOPE YOU’LL PROGRESS THROUGH IT AT WHATEVER STAGE YOU ARE. IF YOU’RE MORE ADVANCED WITH TensorFlow, YOU MIGHT STILL BE INTERESTED IN OUR MASSIVE ONLINE COURSES. AS YOU PROBABLY KNOW, TensorFlow HAS GREAT PARTNERSHIPS WITH DEEP LEARNING CORSERA, AND THESE COURSES ARE AVAILABLE TO EVERYONE IN THIS ROOM, TO EVERYONE WATCHING ONLINE. WE HOPE THAT YOU’LL TAKE THE STUFF THAT WE HAVE IN THESE COURSES, WHICH IS BOTH FROM TensorFlow INSTRUCTORS AND RENOWNED ACADEMIC INSTRUCTORS. AND WE WANT TO GIVE EVERYONE AMPLE OPPORTUNITY TO LEARN TensorFlow. AS YOU HEARD THIS MORNING, THERE IS A NEW SPECIALIZATION ON COURSERA FOR TAKING MODELING, AND DEPLOYING IT IN APPLICATIONS. AND AGAIN, RIGHT, AS YOU MOVE UP THESE STEPS OF KNOWING TensorFlow, WE HOPE YOU’LL CHECK OUT OUR NEW AND UPDATED TUTORIALS AND GUIDES. THIS IS THANKS TO THE AMAZING WORK ON OUR TensorFlow DEVELOPER RELATIONS TEAM. THEY’RE CONSTANTLY WRITING NEW DOCUMENTATION, NEW GUIDES, NEW TUTORIALS. WITH THE LAUNCH OF TF 2.0 ALL OF THESE NEW GUIDES ARE AVAILABLE FOR YOU TO CHECK OUT TF 2.0 AND UNDERSTAND HOW TO USE KARAS AND THE USE CASES. WE HOPE YOU’LL TAKE ADVANTAGE OF THESE RESOURCES THAT WE PROVIDE. AND FINALLY, LET’S JUMP INTO HOW TO GET INVOLVED WITH CONTRIBUTING. SO NOW YOU KNOW TensorFlow, RIGHT? YOU’RE ADVANCED, YOU’VE DEPLOYED, YOU WANT TO CONTRIBUTE TO THE OPEN SOURCE COMMUNITY. WELL, ONE OF THE FIRST WAYS THAT EVERYONE KIND OF THINKS ABOUT IS CONTRIBUTING CODE. AND WE’RE HAPPY TO DESCRIBE TO YOU A WAY THAT WE USE — ON THE TensorFlow TEAM TO CONSULT WIDELY WITH DESIGN, API DESIGNS, AND COMMUNITY MEMBERS, THE REQUEST FOR COMMENTS OR RFCs. THIS IS THE MAIN WAY WE COMMUNICATE CHANGES TO OUR APIs AND RECEIVE DESIGN FEEDBACK. WE’D LOVE TO INVITE EVERYONE HERE TO TAKE A LOOK AND ALSO JOIN. THIS IS ONE EXAMPLE OF AN RFC. THIS IS AN IMPROVED FOREST ESTIMATOR. AND I WOULD LIKE TO TAKE THIS OPPORTUNITY TO OF COURSE THANK EVERYONE WHO HAS AUTHORED OR VIEWED AN RFC. AND WE ACTUALLY HAVE 45 ACCEPTED TO DATE, WHICH IS AN INCREDIBLE NUMBER. AND THEY HAVE RANGE FROM TFS TO T LITE, AND EACH RFC EXPANDS THE USAGE OF TensorFlow. IT HELPS THE COMMUNITY AND IT ALSO IS A GREAT BOON TO THE TensorFlow TEAM. SO, WE’D LOVE TO HAVE YOU ALSO PROPOSE DESIGNS. YOU CAN CHECK OUT MORE ABOUT RFCs AND TALK TO ANY OF US ABOUT THIS, ALSO.>>AND ALSO, FOR BIGGER PROJECTS IN WHICH WE HAVE TO WORK AS A TEAM, WE’VE CREATED THE SPECIAL INTEREST GROUPS, WHICH IS A PROGRAM THAT ORGANIZES THE CONTRIBUTORS INTO MORE FOCUSED STREAMS OF WORK. EVERYTHING STARTED WITH THE SIG BUILD AND NOWADAYS WE HAVE 11SIGs, WHICH IS REALLY IMPRESSIVE HOW THE SIGs HAVE GROWN SO MUCH OVER THE PAST FEW YEARS. SO, ALL THE CONTRIBUTORS — YOU — ARE VERY WELCOME TO JOIN THE SIGs AND REALLY JOIN THE SIG THAT RESONATES MORE WITH THE PARTS THAT YOU EITHER ENJOY OR CARE THE MOST ABOUT TENSORFLOW. JUST AN OVERVIEW OF OUR CONTRIBUTOR ECOSYSTEM, AS YOU CAN SEE, HIGHLIGHTED IN THE DARKER ORANGE, WE HAVE THE SIGGED A-ONS, THE SIG BUILD, IO, NETWORKING, GBM, MICRO-AND RUST, WHICH ARE COMMUNITY-LED OPEN-SOURCE SIGS, AND THE OTHERS, WHICH INCLUDE TENSOR BOARD ARE GOOGLE-LED WITH AN OPEN DESIGN PHILOSOPHY. SO IF YOU SEE A SIG THAT RESONATES WITH THE WORK THAT YOU DO OR IF YOU CARE ABOUT A TOPIC AND WOULD LOVE TO LEARN MORE, THE SIGs HAVE MONTHLY OR WEEKLY CALLS AND YOU ARE VERY WELCOME TO JOIN AS WELL. I WOULD LIKE TO GIVE YOU AN OVERVIEW OF OUR OPEN SOURCE COMMUNITY-LED SIGs, AND JUST BRIEFLY GOING THROUGH SOME OF THE KEY ASPECTS OF THE SIGs, THE SIG ADD-ONS. IT MAINTAINS IMPORTANT ADDITIONS AND ADOPTED SOME OF THE PARTS OF TF CONTRIB. THIS IS LED BY SEAN AND XIU. THE SIG BUILD, WE HAVE ONE OF THE LEADS HERE WITH US TODAY. ACTUALLY, FOCUS ON BUILDING AND PACKAGE TensorFlow FOR DIFFERENT DISTRIBUTION ENVIRONMENTS AND IS LED BY JASON AND AUSTIN ANDERSON. THE SIG IO FOCUSES ON SUPPORTING EXTRA FILE SYSTEMS AND FILE FORMATS FOR TensorFlow. AND IT’S AN INITIATIVE LED BY YOUNG AND ANTON. AND AS WE ALL KNOW, HIGH-PERFORMANCE COMPUTING RESOURCES REQUIRE LIGHTNING FAST INTER-CONNECTIVITY. AND THE SIG NETWORKING FOCUSES ON BUILDING MORE NETWORK SUPPORT FOR TensorFlow. AND THIS IS AN INITIATIVE LED BY TWO PEOPLE. AND FINALLY, THE SIG KARAS. WE’VE ADDED THIS SIG TO IMPROVE THE KARAS API FOR TensorFlow. SO THOSE ARE SOME OF THE SIGs THAT YOU CAN JOIN, BUT WE ALSO HAVE, LIKE I MENTIONED BEFORE, THE OTHER SIGs THAT ARE GOOGLE-LED WITH AN OPEN PHILOSOPHY. YOU’RE VERY WELCOME TO HAVE A LOOK AT THE SIG PLAYBOOK AT TensorFlow.ORG WHERE YOU’LL FIND MORE INFORMATION ON HOW YOU CAN JOIN THE SIGs AND THE ONGOING PROJECTS THAT THEY HAVE RIGHT NOW. IF YOU SEE THAT NONE OF THE SIGs THAT CURRENTLY EXIST ARE A FIT FOR YOU OR FOR YOUR WORK, IF WE SEE THERE’S ENOUGH EVIDENCE AND ENOUGH SUPPORT FROM THE COMMUNITY, YOU CAN ALSO START AND ESTABLISH YOUR OWN SIG. AND IF YOU HAVE POP ON OUR COMMUNITY RESOURCES, THAT’S WHERE YOU WILL SEE HOW THE SIGs OPERATE, WHAT ARE THE RESOURCES AND TOOLS THAT ARE AVAILABLE FOR YOU TO HELP YOU THROUGHOUT THIS PROCESS, BUT ALSO WE HAVE MORE INFORMATION NOT ONLY ABOUT THE SIGs, BUT ALSO OUR RFC PROCESS AND OUR CODE OF CONDUCT. SO I STRONGLY ENCOURAGE YOU TO HAVE A LOOK AT THE TensorFlow WORLD. AND TODAY I’M ALSO EXTREMELY EXCITED TO ANNOUNCE THAT WE HAVE ACHIEVED ANOTHER MILESTONE WITH TensorFlow AND OUR COMMUNITY. WE HAVE HOSTED THE FIRST CONTRIBUTOR SUMMIT JUST ON MONDAY AND TUESDAY FOR ALMOST 100 PARTICIPANTS. AND IT WAS A GREAT WAY TO REALLY CONNECT WITH THE SIG LEADS AND WITH THE BROADER COMMUNITY, AND TO REALLY UNDERSTAND HOW TOGETHER WE CAN MOVE FORWARD WITH THE OPEN SOURCE PROJECT, WHAT ARE THE STRATEGIC DEVELOPMENTS THAT WE CAN IMPLEMENT IN TensorFlow, WHAT AE THE NEEDS, PROJECT MANAGEMENT, COMMUNITY MANAGEMENT. IT WAS A GREAT CONVERSATION THAT WE HAD OVER TWO DAYS. SO, I STRONGLY ENCOURAGE YOU TO, IF YOU DIDN’T HAVE THE CHANCE TO PARTICIPATE THIS TIME, TO HAVE A LOOK AT THE ONLINE RESOURCES THAT WILL BE AVAILABLE AFTERWARDS. IT WAS A GREAT OPPORTUNITY TO CONNECT WITH YOU ALL.>>Nicole: AWESOME. SO, SOME OF THE SIGs, LIKE JOANA MENTIONED, ARE LED BY WHAT WE CALL MACHINE LEARNING GOOGLE DEVELOPERS EXPERTS. I’D LOVE TO SHOW YOU A LITTLE BIT ABOUT WHAT THAT MEANS. SO OUR ML GDEs ARE A GLOBAL NETWORK OF ML EXPERTS THAT GOOGLE WORKS CLOSELY WITH. WE PROVIDE THE LATEST INFORMATION TO THEM. THEY GIVE US FEEDBACK. IT’S AN AWESOME RELATIONSHIP. WE’RE EXCITED. WE HAVE 126 OF THEM TO DATE WORLDWIDE. THIS YEAR ALONE, THE MLGDs HAVE GIVEN OVER 400 TALKS, HOSTED OVER 250 WORKSHOPS AND WRITTEN OVER 200 ARTICLES. THIS IS INCREDIBLE BECAUSE WE ACTUALLY KNOW THAT THESE TALKS, WORKSHOPS AND ARTICLES HAVE REACHED A WORLDWIDE AUDIENCE OF 435,000 DEVELOPERS. AS YOU CAN IMAGINE, TENSORFLOW, OUR TEAM, WE WANT TO REACH AS MANY PEOPLE AS WE CAN. BUT WITH THE MLGDs WE AMPLIFY THAT REACH OF IMPACT THAT WE CAN HAVE IN THE WORLD OF TEACHING IT AT TensorFlow, AND REALLY HELPING PEOPLE AROUND THE WORLD UNDERSTAND ABOUT TensorFlow. SO, WE’RE REALLY EXCITED. WE WOULD LOVE TO TELL YOU — IF YOU WANT TO BECOME A GDE, THIS IS A LINK TO BECOME A GDE. WE HAVE A LOT OF LINKS FOR CONNECTING WITH OTHER GDEs H. .TODAY WE WOULD LOVE TO WELCOME ONE OF OUR GDEs TO THE STAGE TO GIVE A BRIEF STAGE WITH US. SO, PLEASE WELCOME JASON DEVON.>>HI, EVERYONE. I’M A COMMUNITY LEAD FOR SIG BUILD. WE HAVE A FEW MEMBERS HERE. THANK YOU. AND TO BUILD BEING THE FIRST SIG, IT WAS FORMED TWO YEARS AGO? QUITE A WHILE. SO I’VE REALLY S SEEN THE COMMUNITY GROW A LOT IN THAT TIME. WE HAVE SO MANY SIGs DOING ALL KINDS OF THINGS. AND IT — I STARTED BUILD BECAUSE I SAW PROBLEMS WHEN I WAS TRYING TO USE IT AND I WANTED TO MAKE IT BETTER. AND REALLY, THE GROUP HAS GROWN. AND DONE A LOT OF GREAT THINGS. I WANT TO ENCOURAGE EVERYONE TO GET INVOLVED. YOU CAN JOIN A SIG THAT ALREADY EXISTS. YOU CAN FIND A THING YOU WANT TO DO, WORK ON IT, FIND MORE PEOPLE THAT ARE ALSO INTERESTED IN IT, MAYBE START A NEW SIG. A LOT OF PEOPLE AROUND TO HELP. THESE PEOPLE ARE WONDERFUL. AND I’M ALSO ONE OF THE MLGDs, SO IT’S A GREAT PROGRAM. IT’S REALLY NICE TO HEAR FROM OTHER MLGDEs, THEY WORK ON ALL KINDS OF CUTTING-EDGE STUFF, ALL KINDS OF DIFFERENT FIELDS, STUFF THAT I DON’T KNOW OR HEAR ABOUT OTHER THAN THEM. SO, REALLY GOOD. YEAH. THANK YOU. [ APPLAUSE ]>>THANK YOU.>>THANK YOU SO MUCH, JASON. AND WE’RE REALLY LUCKY TO HAVE ANOTHER MLGDE IN THE AUDIENCE. PLEASE WELCOME MARGARET.>>HELLO, EVERYONE. I’M A MACHINE LEARNING GDE. I’M ALSO THE LEAD ORGANIZER OF GOOGLE DEVELOPER GROUP SEATTLE AND ANOTHER GROUP CALLED THE SEATTLE DATA ANALYTICS AND MACHINE LEARNING. I BECAME A MACHINE LEARNING GDE IN 2018. AND HERE IS WHY I LOVE BEING PART OF THIS AMAZING COMMUNITY. I GET TO COLLABORATE WITH OTHER MACHINE LEARNING GDEs AND GOOGLERS ON VARIOUS PROJECTS. FOR EXAMPLE, I GET TO WRITE SOME TUTORIALS THAT YOU WILL FIND ON TensorFlow.ORG AND SOME OF THE BLOG POSTS THAT WERE PUBLISHED ON THE TensorFlow MEDIUM PUBLICATION. AND EARLIER THIS YEAR, I HELPED ORGANIZING THE GLOBAL TensorFlow DOG SPRINT WITH PAGE, SERGEI, AND OTHER MACHINE LEARNING GDEs AND GDE ORGANIZERS. IT WAS AN INCREDIBLE EXPERIENCE TO WORK ON SUCH A HIGH-IMPACT PROJECT WHICH WAS EVEN MENTIONED IN THE KEYNOTE THIS MORNING. SO, I SPEAK ABOUT TensorFlow AND MACHINE LEARNING AT VARIOUS CONFERENCES AND I REALLY ENJOY THE OPPORTUNITY TO BE ABLE TO PREVIEW GOOGLE PRODUCTS AND PROVIDE FEEDBACK. SO MANY OF THE MACHINE LEARNING GDEs ARE WELL-KNOWN EDUCATORS, SPEAKERS, OR O’REILLY BOOK AUTHORS. IT’S REALLY GREAT TO BE ABLE TO LEARN FROM MY FELLOW GDEs AND GOOGLERS. AND ONCE A YEAR WE WILL GATHER TOGETHER FOR OUR GLOBAL GDE SUMMIT AROUND THE WORLD, THE GDEs FROM AROUND THE WORLD. WE JUST HAD THE SUMMIT A FEW DAYS AGO BEFORE TensorFlow WORLD. SO TO BECOME A GDE, MACHINE LEARNING GDE IN PARTICULAR, YOU NEED TO BE ABLE TO DEMONSTRATE BOTH YOUR COMMUNITY CONTRIBUTION AS WELL AS KNOWLEDGE IN MACHINE LEARNING. WE WOULD LOVE TO SEE MORE OF YOU JOIN OUR GROWING MACHINE LEARNING GDE COMMUNITY. THANK YOU. [ APPLAUSE ]>>Joana: THANK YOU SO MUCH. THIS IS FANTASTIC. I AM SURE I CAN SPEAK FOR BOTH OF US. I’M ALWAYS SO IMPRESSED BY THE FANTASTIC AND AMAZING WORK THAT OUR GDEs DO. IT’S REALLY NICE TO SEE HOW ENGAGED THE COMMUNITY IS. HOWEVER, THERE’S MANY OTHER WAYS BY WHICH YOU CAN CONTRIBUTE TO TensorFlow. IT DOESN’T HAVE TO BE ONLY THROUGH CODE. SO, IF YOU ARE A CODER, BUT YOU WOULD LIKE TO LEARN OR DEVELOP A NEW SKILL SET THERE’S MANY OTHER WAYS THAT YOU CAN GET INVOLVED WITH TensorFlow. SO, WHEN IT COMES DOWN TO CODE CONTRIBUTIONS, THERE ARE THREE MAIN PILLARS THAT WE NORMALLY ENCOURAGE OUR CONTRIBUTORS TO JOIN. PRIMARILY ON USER SUPPORT, WHICH INCLUDES CREATING DOCUMENTATION, TRANSLATION, TRAINING COURSES THAT REALLY WILL HELP OTHER CONTRIBUTORS GETTING INVOLVED AND ONBOARDED WITHIN THE PROJECT. IN TERMS OF COMMUNITY MANAGEMENT, REALLY THROUGH ORGANIZING EVENTS, MEETUPS, AND ALL THE INITIATIVES THAT GET THE COMMUNITY TOGETHER AND ENERGIZED AND EXCITED ABOUT MACHINE LEARNING AND TensorFlow. AND THEN ON THE PROJECT MANAGEMENT SIDE, CREATING THE TOOLS AND RESOURCES THAT WILL HELP ADVANCE OUR PROJECTS BUT ALSO KEEP THE HEALTH AND THE SUSTAINABILITY OF THE INITIATIVES THAT WE DO. SOMETIMES WE WORK REALLY ON CROSS-FUNCTIONAL TEAMS ON BUILDING THE USE CASES ON HOW TensorFlow CAN BE IMPLEMENTED IN DIFFERENT WAYS. AND THEN FINALLY I WOULD LIKE TO HIGHLIGHT THAT WE HAVE A CODE OF CONDUCT IN OUR TensorFlow COMMUNITY. SO WE APPLY THIS CODE OF CONDUCT TO ALL THE EVENTS AND THE INITIATIVES THAT WE DO. AND WE WOULD LIKE TO REMIND YOU THAT THIS IS A SAFE SPACE WHERE YOU CAN TRULY BE YOURSELF AS A CONTRIBUTOR, AND WE WELCOME THAT DIVERSITY OF IDEAS, OPINIONS, AND SUGGESTIONS. SO, IF YOU SEE THAT SOMETHING IS JUST NOT RIGHT, PLEASE FEEL FREE THAT YOU KNOW THAT YOU CAN ESCALATE THOSE PROBLEMS TO ALSO THE COMMUNITY STEWARDS. WE’RE HERE FOR YOU. WE ARE HERE TO MAKE SURE THAT YOU FEEL ENGAGED, THAT YOU FEEL HEARD, AND THAT YOU FEEL THAT YOU BELONG TO A COMMUNITY OF EXCITED MACHINE LEARNING EXPERTS, CONTRIBUTORS, AND USERS.>>Nicole: SO, WE WANT TO WRAP UP OUR CONVERSATION BY REVISITING THE LINKS AND THE DIFFERENT RESOURCES THAT WE HAVE GIVEN YOU IN THIS TALK. SO, AGAIN, AFTER TF WORLD YOU’RE WONDERING HOW DO WE KEEP ONE THH THE LATEST NEWS AND DEEP DIVES FROM TensorFlow. WELL, THESE ARE THE WAYS YOU CAN KEEP UP WITH US. TWITTER IS VERY GREAT FOR A LOT OF THE LATEST ANNOUNCEMENTS AND UPDATES FROM THE TensorFlow TEAM. OUR BLOG IS ACTUALLY AN AMAZING RESOURCE, A LOT OF DEEP DIVES, A LOT OF UNDERSTANDING SPECIFIC USE CASES. YOU MIGHT BE WONDERING HOW TO USE TensorFlow IN A CERTAIN APPLICATION AND THE BLOG MAY ACTUALLY HAVE A GUEST POST OR A POST FROM THE TensorFlow TEAM THAT CAN ADDRESS THAT. SO WE REALLY SUGGEST YOU CHECK OUT THE BLOG. AND YouTube — I THINK MANY OF YOU PROBABLY ALREADY HAVE SEEN THE TensorFlow YouTube CHANNEL. BUT IN CASE YOU HAVEN’T IT’S ACTUALLY A REALLY AWESOME RESOURCE TO LEARN TensorFlow. SO WE HAVE A LOT OF VIDEOS THAT HIGHLIGHT OUR NEW ANNOUNCEMENTS, HOW TO USE TensorFlow, HOW TO USE SPECIFIC THINGS LIKE TF KARAS. WE HAVE VIDEOS ABOUT THAT. AND ONE OF OUR MOST POPULAR VIDEOS IS DONE BY SOMEONE ON OUR DEVELOPER RELATIONS TEAM, THE ML0 TO HERO VIDEO. IT’S A GREAT RESOURCE. WE HIGHLY SUGGEST YOU FOLLOW. THAT’S HOW YOU’LL GET UPDATES FROM TensorFlow OUTSIDE OF TF WORLD. AND FINALLY, THESE ARE SOME OF THE LINKS THAT WE SHOWED EARLIER. WE WANT TO EMPHASIZE, TensorFlow THE COMMUNITY WOULD NOT BE POSSIBLE WITHOUT EVERYBODY IN THE ROOM, EVERYONE IN THE COMMUNITY GLOBALLY. WE ENCOURAGE YOU, IF YOU AREN’T IN A SIG OR A USER GROUP, IF YOU’RE INTERESTED, YOU CAN CHECK OUT EVERYTHING ON OUR TensorFlow.ORG COMMUNITY LINKS. YOU CAN CHECK OUT THE EDUCATIONAL RESOURCES I MENTIONED AT THE BEGINNING. AND WE ARE SO EXCITED THAT SO MANY OF YOU ARE AMONG US IN THE GROUP TODAY. SO WE’D REALLY LOVE TO, YOU KNOW, WELCOME YOU TO ALSO SHARE WITH YOUR FELLOW CONFERENCE ATTENDEES WHAT IT’S LIKE BEING IN A SIG, WHAT IT’S LIKE LEADING A SIG OR BEING AN MLGDE, TOO. SO WITH THAT –>>Joana: WE HAVE OUR CALL TO ACTION TO YOU, WHICH IS JOIN THE USER GROUPS, JOIN THE SIGs, BE PART OF THE COMMUNITY, CONTRIBUTE CODE TO TensorFlow, DOCUMENTATION, TRANSLATIONS, EDUCATIONAL RESOURCES, EVENTS. THERE’S SO MANY DIFFERENT AND EXCITING WAYS TO CONTRIBUTE TO TensorFlow. SO THANK YOU FOR BEING WITH US TODAY. IT’S BEEN REALLY A PLEASURE SPEAKING TO YOU ABOUT THE MANY WAYS THAT YOU CAN GET INVOLVED WITH THE COMMUNITY. AND I HOPE THAT WE CAN CONTINUE THESE CONVERSATIONS. WHAT DO YOU THINK, NICOLE?>>Nicole: THAT SOUNDS PERFECT. LET US KNOW IF YOU HAVE ANY QUESTIONS AND WE’RE SO HAPPY THAT YOU WANT TO BE A PART OF THE TensorFlow COMMUNITY. THANK YOU.>>Joana: THANK YOU. [ APPLAUSE ]>>WELCOME TO THE SANDBOX. WE SAW SOME AMAZING STUFF YOU CAN ACCOMPLISH. I’M HERE TO SHOW HOW YOU CAN BUILD THE SAME STUFF, FROM OBJECT DETECTION TO IMAGE CLASSIFICATION AND VOICE RECOGNITION. WE’RE SHOWING PEOPLE HOW TO DEPLOY THAT. TO SHOW YOU WHAT’S POSSIBLE WITH TensorFlow LITE, WE BUILT THIS AMAZING EXPERIENCE.>>BEHIND ME IS DANCE, A SUPER-FUN APPLICATION USING TensorFlow LIGHT, THE MOBILE EMBEDDED SYSTEMS FRAMEWORK FOR RUNNING MACHINE LEARNING. IT TEACHES PEOPLE HOW TO DANCE BY RUNNING SEGMENTATION ON THE GBU. WE HAVE A BUNCH OF OPS WE JUST RELEASED. IT ENABLES YOU TO RUN SUPERFAST MODELS. WE’RE RUNNING TWO IN REAL TIME AND GIVING THE USER FEEDBACK WHICH HELPS THEM DANCE BETTER. TO BUILD INTERESTING APPLICATIONS ON MOBILE PHONES AND EXECUTING THIS ON-DEVICE, YOU CAN GO TOTensorFlow.ORG/LIVE. THERE’S HEAPS OF SAMPLE CODE, DOCUMENTATION, AND YOU CAN BUILD AN AMAZING APPLICATION LIKE DANCE AND RELEASE IT TO THE WORLD.>>WE’RE LOOKING AT A MODEL CALLED POSE NET, A MODEL THAT’S TRAINED ON IMAGES OF HUMAN BEINGS TO PREDICT THEIR SKELETAL POSE. THIS MODEL IS RUNNING ENTIRELY IN THE BROWSER WITH A LIBRARY CALLED TensorFlow.JS, WHICH IS A MACHINE LEARNING LIBRARY FOR SCRIPT THAT CAN RUN ENTIRELY IN THE BROWSER AND USER GPU THROUGH WEB GL. NOW, IT ALSO RUNS IN NO JS USING THE BINARY AND ALL OF ITS HARDWARE ACCELERATION STORIES. PIANO GENIE GENERATES MIDI NOTES. THIS MODEL BRINGS A HUMAN IN THE LOOP AND LETS THE HUMAN CONDITION WHERE THE MODEL IS ACTUALLY GOING TO GO. ♪ >>TensorFlowJS IS OPEN SOURCE AND IS AVAILABLE ONLINE AT TensorFlow.ORG/JS, AS ARE MULTIPLE MORE EXAMPLES AND DEMOS OF HOW TO USE IT. HEAD OVER THERE TO START BUILDING FOR THE BROWSER.>>WE TAKE AN OLD CELL PHONE AND PUT IT UP IN THE TREES. IT LISTENS TO THE SOUNDS OF THE FOREST TO PICK UP THE SOUND OF ANYTHING THAT INDICATES THERE’S SOME DANGEROUS. DANGER.>>[ SPEAKING NON-ENGLISH LANGUAGE ]>>WOULD IT BE EASIER FOR US TO DO THAT BY FOCUSING ON THE>>[ SPEAKING NON-ENGLISH LANGUAGE ]>>THEY ARE AN AMAZING EXAMPLE OF A PEOPLE TRYING TO PRESERVE THEIR FOREST. THEY’RE WELL-EDUCATED AND LOOKING FOR COLLABORATION, NOT HELP. PEOPLE ON THE GROUND IN THESE PLACES ARE THE SOLUTION. THEY’RE THE ONES WHO CAN FIGHT OFF DEFOREST FORESTATION. TECHNOLOGY CAN HELP THEM DO IT MORE SAFELY AND MORE>>[ SPEAKING NON-ENGLISH LANGUAGE ]>>ALL HE HAS TO DO IS SPEAK ON THIS ONE HERE, BECAUSE THE ANTENNA IS MORE FRAGILE AND SENSITIVE. YOU WANT TO PUT THE MICROPHONE ON THERE. YEAH, THAT’S GOOD. OLD CELL PHONES ARE AVAILABLE TO ALMOST ANYBODY ALL THE TIME. THESE AREN’T TRASH. THESE ARE ACTUALLY REALLY POWERFUL COMPUTERS THAT CAN CONNECT TO THE NETWORKS THAT ARE THERE. THEY CAN RECORD SOUND AND DO PROCESSING. THE HARD PARTS HAVE TO DO WITH MAKING IT POSSIBLE TO POWER THEM AND PICK UP SOUNDS FROM A GREAT DISTANCE. THESE ARE THINGS YOU CAN DO WITH STANDARD ELECTRONICS.>>THEY HAVE AT MOST 30 RANGERS, 30 WARRIORS THAT ARE ABLE TO PATROL THE AREA. WE’RE TALKING ABOUT AREA THAT’S HUGE. THAT’S WHERE THIS ABILITY TO LISTEN TO PARTS OF THE FOREST INCREASES THE TINTERCY AND — EFFICIENCY AND SAFETY OF THE OPERATION FOR THEM. A PERSON COULD BE DETECT A CHAINSAW AT A KILOMETER AWAY, GIVEN HOW COMPLEX THE SOUND IS. WE’RE USING AN OPEN SOURCE MACHINE LEARNING TOOL, TensorFlow, THAT’S ABLE TO DETECT THE SOUNDS OF BIRDS, ANIMALS, OR CHAINSAWS. IT’S AMAZING WHAT THEY CAN FIND THAT WE CAN’T HEAR. IT CAN SEND AN ALERT SO THE CHIEFS CAN DECIDE HOW TO>>[ SPEAKING NON-ENGLISH LANGUAGE ] ♪ >>HERE WE ARE, 5:00. LAST SESSION OF THE DAY. HOW’S YOUR ENERGY? GOOD? THIS IS THE TALK YOU’VE ALL BEEN WAITING FOR. ZZAK STONE, PROJECT MANAGER FOR CLOUD HAS AN AWESOME TALK ABOUT RESEARCH PROJECTS IN THE CLOUD. HERE YOU ARE, ZAK. [ APPLAUSE ]>>Zak: THANK YOU VERY MUCH. I’M DELIGHTED TO BE HERE TODAY TO TALK TO YOU ABOUT SOME OF THE FANTASTIC TensorFlow RESEARCH CLOUD PROJECTS AND TO INVITE YOU TO START YOUR OWN, WHETHER YOU’RE IN THE ROOM, ON THE LIVE STREAM, OR WATCHING AFTERWARDS. YOU’RE WELCOME TO GET INVOLVED WITH TFRC. JUST VERY BRIEFLY SINCE I’M SURE YOU’VE HEARD THIS ALL TODAY, THE CONTEXT IS THIS MASSIVE IMPROVEMENT IN COMPUTATIONAL CAPABILITIES DRIVEN BY DEEP LEARNING, AND SPECIFICALLY, DEEP NEURAL NETWORKS, ENABLING EXCITING APPLICATIONS, TOUCHING ALL KINDS OF DIFFERENT RANGING FROM IMAGES TO SPEECH, TO TEXT, EVEN FULL SCENES. AND THE CHALLENGE THAT MANY OF YOU ARE PROBABLY GRAPPLING WITH IS THAT THESE NEW CAPABILITIES COME WITH PROFOUND INCREASES IN COMPUTE REQUIREMENT. A WHILE BACK, OPEN AI DID A STUDY WHERE THEY MEASURED THE TOTAL AMOUNT OF COMPUTE REQUIRED TO TRAIN SOME OF THESE FAMOUS MACHINE LEARNING MODELS OVER THE PAST SEVERAL YEARS AND THE IMPORTANT THING IS THAT IT’S ACTUALLY A LONG SCALE ON THE COMPUTE AXIS. THERE ARE TREMENDOUS INCREASES IN THE TOTAL AMOUNT OF COMPUTE REQUIRED TO TRAIN THESE MODELS OVER TIME. AND THERE’S THIS CONSISTENT TREND THAT THESE NEW CAPABILITIES ARE BEING UNLOCKED BY THE ADDITIONAL COMPUTE POWER AS WELL AS LOTS OF HARD WORK BY MANY RESEARCHERS ALL AROUND THE WORLD IN THIS OPEN COMMUNITY. SO UNFORTUNATELY, THESE TREMENDOUS DEMANDS FOR COMPUTE TO MEET THESE NEW OPPORTUNITIES OPENED UP BY DEEP LEARNING ARE COMING TO US JUST AS MOORE’S LAW IS ENDING. WE’VE BENEFITTED FOR DECADES UPON DECADES IN CONSISTENT INCREASES IN SINGLE-THREADED CPU PERFORMANCE, BUT ALL OF A SUDDEN NOW WE’RE DOWN TO MAYBE 3% PER YEAR. WHO KNOWS. THERE COULD ALWAYS BE A BREAKTHROUGH BUT WE’RE NOT EXPECTING EXTRAORDINARY YEAR UPON YEAR GAINS AS WE’VE ENJOYED IN THE PAST. IN RESPONSE TO THAT, WE BELIEVE THAT SPECIALIZED HARDWARE FOR MACHINE LEARNING IS THE PATH FORWARD FOR MAJOR PERFORMANCE WINS, COST SAVINGS AND NEW BREAKTHROUGHS ACROSS ALL THESE RESEARCH DOMAINS THAT I MENTIONED EARLIER. NOW, AT GOOGLE, WE HAVE DEVELOPED A FAMILY OF SPECIAL PURPOSE MACHINE LEARNING ACCELERATORS CALLED CLOUD TBUs. WE’RE ON OUR THIRD GENERATION. TWO ARE AVAILABLE IN THE CLOUD, THE SECOND AND THE THIRD. TO GIVE YOU A BRIEF OVERVIEW OF THE HARDWARE I’M GOING TO BE TALKING ABOUT, WE HAVE THESE INDIVIDUAL DEVICES HERE CALLED 2 AND 3. WE’VE MADE TREMENDOUS PROGRESS GENERATION OVER GENERATION — 180 TO 420. WE’VE INCREASED THE MEMORY FROM 64 TO 128 WHICH MATTERS A LOT IF YOU CARE ABOUT CUTTING-EDGE NATURAL LANGUAGE PROCESSING MODELS. BUT THE MOST IMPORTANT THING ISN’T JUST THESE INDIVIDUAL DEVICES, WHICH ARE THE BOARDS THAT YOU SEE HERE WITH THE CHIPS CONNECTED TO A CPU HOST THAT’S NOT SHOWN. IT’S THE FACT THAT THESE DEVICES ARE DESIGNED TO BE CONNECTED TOGETHER INTO MULTI-RACK MACHINE LEARNING SUPERCOMPUTERS THAT LET YOU SCALE MUCH FURTHER AND PROGRAM THE WHOLE SUPERCOMPUTER ACROSS THESE MANY RACKS AS IF IT WERE A SINGLE MACHINE. ON THE TOP HERE YOU CAN SEE THE POD SPANNING FOUR RACKS. THE TPUs ARE IN THE CENTER CLUMN. THE CPUS ARE ON THE OUTSIDE. THAT MACHINE GOT US TO 11.5 PETAFLOPS. THE TPU CHIPS IN PARTICULAR ARE CONNECTED BY THE 2D NETWORK THAT ENABLES ULTRA FAST COMMUNICATION, MUST FASTER THAN STANDARD NETWORK. THAT’S A BIG FACTOR, ESPECIALLY IF YOU CARE ABOUT MODEL PARALLELISM AND PARTITIONING. NOW, WITH THE POD, WHICH IS LIQUID COOLED, IT SPANS EIGHT RACKS AND GETS YOU UP OVER 100 PETAFLOPS IF YOU’RE USING THE ENTIRE MACHINE SIMULTANEOUSLY. ON A RAW OP BY OP BASIS, IT’S COMPETITIVE WITH THE LARGEST SUPERCOMPUTERS IN THE WORLD, ALTHOUGH THESE USE LOWER PRECISION, WHICH IS APPROPRIATE FOR DEEP LEARNING. I’VE MENTIONED PERFORMANCE. I WANTED TO QUANTIFY THAT BRIEFLY IN THE MOST RECENT TRAINING VERSION 0.6 COMPETITION, CLOUD TPUs WERE ABLE TO OUTPERFORM ON-PREM INFRASTRUCTURE. YOU CAN SEE THE TPU RESULTS IN BLUE COMPARED WITH THE LARGEST ON-PREMISE CLUSTER RESULTS SUBMITTED IN THE COMPETITION. IN THREE OF THE FIVE CATEGORIES WE ENTERED, THE CLOUD TPUs DELIVERED THE BEST TOP-LINE RESULTS, INCLUDING 84% INCREASES OVER THE NEXT ENTRY IN MACHINE TRANSLATION, BASED ON TRANSFORMER, AND OBJECT DETECTION. NOW, THESE NUMBERS ARE EVOLVING ALL THE TIME. THERE’S TREMENDOUS INVESTMENT AND PROGRESS IN THE FIELD. I WANTED TO ASSURE YOU THESE TPUs CAN DELIVER WHEN IT COMES TO HIGH PERFORMANCE. BUT TODAY WE’RE HERE TO TALK ABOUT RESEARCH AND EXPANDING ACCESS TO THIS TREMENDOUS POWER TO EDGE ABLE — ENABLE RESEARCHERS TO BENEFIT, MAKE THEIR CONTRIBUTIONS TO EXPAND THE FRONTIER. IN ORDER TO INCREASE ACCESS TO CUTTING-EDGE MACHINE LEARNING COMPUTE, WE’RE THRILLED TO HAVE CREATED THE CLOUD IN ORDER TO ACCELERATE OPEN MACHINE LEARNING RESEARCH AND DRIVE THIS FEEDBACK CYCLE WHERE MORE PEOPLE THAN EVER BEFORE HAVE ACCESS TO THEY HAVE BREAKTHROUGHS. THEY PUBLISH PAPERS AND BLOG POSTS AND OPEN SOURCE CODE AND GIVE TALKS AND SHARE THE RESULTS WITH OTHERS. THAT HELPS MORE PEOPLE GAIN ACCESS AND BENEFIT. WE’RE TRYING TO DRIVE THIS POSITIVE FEEDBACK LOOP. WE’VE ACTUALLY MADE WELL OVER 1,000 OF THESE CLOUD TPU DEVICES AVAILABLE FOR FREE TO SUPPORT THIS OPEN MACHINE LEARNING RESEARCH. IF YOU’RE INTERESTED IN LEARNING MORE, I’LL HAVE MORE INFORMATION AT THE END OF THE TALK. THIS POOL OF COMPUTE, THE TFRC CLUSTER, INVOLVES NOT JUST THE ORIGINAL TPU2 DEVICES, BUT WE’VE ADDED SOME OF THE V3 DEVICES OF THE LATEST GENERATION, IF YOU’RE PUSHING THE LIMITS. AND THERE’S THE POTENTIAL FOR CLOUD POD ACCESS. IF YOU’VE GONE AS FAR AS YOU CAN WITH THE INDIVIDUAL DEVICES, LET US N KNOW. THE UNDERLYING MOTIVATION IS THE OBSERVATION THAT TALENT IS EQUALLY DISTRIBUTED THROUGHOUT THE WORLD BUT OPPORTUNITY IS NOT. WE’RE TRYING TO CHANGE THAT BALANCE TO MAKE MORE OPPORTUNITIES AVAILABLE TO TALENTED PEOPLE ALL AROUND THE WORLD WHEREVER THEY MIGHT BE. SO, WE’VE HAD TREMENDOUS INTEREST IN THE TFRC PROGRAM SO FAR. MORE THAN 26,000 PEOPLE HAVE CONTACTED US. AND WE’RE THRILLED THAT WE’VE BEEN ABLE TO ONBOARD MORE THAN 1,250 RESEARCHERS AND WE’RE ADDING MORE RESEARCHERS ALL THE TIME. IF YOU HAVEN’T HEARD FROM US YET, PLEASE PING US AGAIN. WE WANT TO SUPPORT YOU WITH TFRC. THE FEEDBACK LOOP IS JUST STARTING TO TURN, BUT ALREADY I’M HAPPY TO ANNOUNCE THAT MORE THAN 30 PAPERS IN THE ACADEMIC COMMUNITY HAVE BEEN ENABLED BY TFRC. MANY OF THESE RESEARCHERS TELL US WITHOUT TFRC COMPUTE, THEY COULDN’T HAVE AFFORDED TO CARRY OUT THIS RESEARCH. LEVER OF PROGRESS AND HAVE TIPPED IT SLIGHTLY UPWARDS SO THE WHOLE FIELD IS MOVING JUST A LITTLE BIT FASTER. WE REALLY THANK YOU ALL FOR BEING PART OF THAT. I’M MOST EXCITED TO SHARE SOME OF THE STORIES DIRECTLY OF THE INDIVIDUAL RESEARCHERS AND THE PROJECTS THAT THEY’VE BEEN CARE CARRYING OUT ON THE TFRC CLOUD TPUs. THESE RESEARCHERS COME FROM ALL OVER THE WORLD. I ONLY HAVE TIME TO HIGHLIGHT FOUR PROJECTS TODAY. BUT THE FANTASTIC THING IS THAT THREE OF THESE RESEARCHERS HAVE BEEN ABLE TO COME AND TRAVEL HERE TO BE WITH US IN PERSON. SO YOU’LL GET TO HEAR ABOUT THEIR PROJECTS IN THEIR OWN WORDS. WE’LL START WITH VICTOR HERE IN THE UPPER LEFT. WELCOME, VICTOR. COME ON UP. [ APPLAUSE ]>>victor: HELLO, EVERYONE. I’M EXCITED TO BE HERE. I’M VICTOR, ORIGINALLY FROM NIGERIA. AND CURRENTLY I’M A RESEARCH ENGINEER WITH LABS IN BROOKLYN, NEW YORK. AND SO ABOUT A YEAR AGO I GOT REALLY FASCINATED ABOUT THIS WHOLE AREA OF THE INTERSECTION OF AI. GIVEN MY BACKGROUND IN HUMAN-COMPUTING INTERACTION, IT WAS SOMETHING I REALLY WANTED TO DO. RIGHT ABOUT THAT TIME I GOT THE OPPORTUNITY TO HAVE ACCESS TO THE TFRC. AND TODAY I’M GOING TO TALK TO YOU ABOUT THE RESULTS OF THOSE EXPERIMENTS AND THE RESEARCH RESULTS I HAD WORKING ON THIS. WHY DID I WORK ON THIS PROJECT? AS A LITTLE KID GROWING UP IN EASTERN NIGERIA, MY EXTENDED FAMILY AND I WOULD TRAVEL TO OUR VILLAGE ONCE A YEAR. AND ONE OF THE INTERESTING CAPTIVATING PARTS OF THOSE TRIPS WAS SOMETHING CALLED THE EAST END GRID DANCES OF AFRICA. WHAT WOULD HAPPEN IS THERE ARE DANCERS WITH ELABORATE MASKS. AND RIGHT AS A KID I WAS FASCINATED. THIS PROJECT WAS A WAY TO BRIDGE MY INTEREST IN TECHNOLOGY, ART, AND AS A RESEARCH ENGINEER TO EXPRESS MY IDENTITY THROUGH A PROJECT LIKE THIS. IN ADDITION TO THIS, THERE’S A GROWING AREA OF AI-INSPIRED ART OR AI-GENERATED ART. MOST OF THE DATASETS USED, MAINLY CLASSICAL EUROPEAN ART, REMBRANDT, PI CASS SEW. A PROJECT LIKE THIS IS A WAY TO DIVERSIFY THE CONVERSATIONS IN THAT AREA. FINALLY, THE RESEARCHERS WORKING IN THE GENERATIVE MODEL — ONE OF THE GOALS HERE IS TO CONTRIBUTE A MORE COMPLEX DATA SET COMPARED TO THINGS LIKE FACES. AND IT CAN BE A REALLY INTERESTING WAY TO BENCHMARK SOME OF THE REGENERATED MODELS THAT ARE BEING RESEARCHED TODAY. SO WHAT DID I DO? LIKE ALL OF US KNOW, THE BEST RESULTS — MOST OF THE EFFORT FROM PROJECTS COMES FROM THE DATA COLLECTION PHASE. I STARTED OUT COLLECTING IMAGES. I CURATED 20,000 AND GOT IT DOWN TO 9,000 HIGH-QUALITY IMAGES. I WAS READY TO TRAIN MY MODEL. THE BEAUTIFUL THING IS THE TensorFlow TEAM HAD MADE AVAILABLE A COUPLE OF REFERENCE MODELS. I STARTED MY EXPERIMENT USING AT IMPLEMENTATION USING THE TensorFlow TPS. THE PICTURE YOU SEE ON THE RIGHT IS A VISUALIZATION OF THE TRAINING PROCESS. SO THAT’S RANDOM NOISE. AND AS TRAINING PROGRESS, IT LEARNS TO GENERATE IMAGES THAT ARE SIMILAR TO THE DISTRIBUTION. AND SO STARTING OUT WITH A REFERENCE IMPLEMENTATION, THERE ARE TWO INTERESTING THINGS I DID. FIRST WAS TO MODIFY THE CONFIGURATION OF THE NETWORK, THE PARAMETERS TO LET IT READ LARGER IMAGES. AND THEN THE PIPELINE LETS ME FEED MY DATA SET INTO THE MODEL. SO THE THING YOU SHOULD WATCH OUT FOR IS TO ENSURE THAT YOUR DATA IN THE PIPELINE MATCHES WHAT THE R REFERENCE MODEL IS EXPECTING. IT TOOK ME 60 EXPERIMENTS TO TRACK DOWN THAT ERROR AND FIX IT. IT DID TAKE A COUPLE OF DAYS. I RAN ABOUT 200 EXPERIMENTS. AT THIS POINT THIS IS WHERE TensorFlow RESEARCH CLOUD MAKES A DIFFERENCE. SOMETHING LIKE THIS WOULD TAKE A COUPLE OF WEEKS TO GET DONE. BUT I WAS ABLE TO RUN THESE EXPERIMENTS, ONCE ALL BUGS WERE FIXED, WITHIN ONE OR TWO DAYS. AT THIS POINT ALL THE IMAGES LOOK LIKE MASKS. BUT NONE OF THEM ARE REAL. NONE OF THEM EXIST IN THE REAL WORLD. THESE ARE ALL ARTISTIC INTERPRETATIONS OF WHAT AN AFRICAN MASK WOULD LOOK LIKE. WHAT COULD I DO NEXT? I STARTED TO THINK, AT THIS POINT I HAVE A MODEL THAT DOES PRETTY WELL BUT THE QUESTION IS, ARE THE IMAGES NOVEL, ARE THEY NEW STUFF OR HAS THE MODEL MEMORIZED INPUT DATA AND REGURGITATED THAT? TO ANSWER THOSE QUESTIONS, I TOOK A SEMANTIC SEARCH APPROACH WHERE I USED A PRETRAINED MODEL TO ACCESS FEATURES FROM THE DATA SETS AND GENERATED IMAGES AND BUILT THIS INTERFACE THAT ALLOWS SOME SORT OF ALGORITHMIC INSPECTION WHERE FOR EACH GENERATED IMAGE I CAN FIND THE TOP 20 IN THE DATA SET. SO THIS IS ONE WAY TO INSPECT THE RESULTS. GOING FORWARD, THE BEST STABLE MODEL I WAS ABLE TO TRAIN COULD ONLY GENERATE 64 IMAGES. CAN WE DO BETTER? YOU CAN USE SUPER-RESOLUTION. THS IS ONE OF MY FAVORITE RESULTS WHERE WE HAVE A SUPER-RES LOTION GAN FROM THE PIXEL AI MODEL. AND HERE IS ANOTHER INTERESTING RESULT. AND WHAT YOU PROBABLY CAN’T SEE VERY CLEARLY HERE IS THAT THERE’S DETAIL IN THE IMAGE THAT DOES NOT EXIST IN THE LOWER RESOLUTION IMAGE. IT’S LIKE TWO-STEP INTERPRETATION USING NEURAL NETWORKS. IF YOU’RE A RESEARCHER, ARTIST OR SOFTWARE ENGINEER INTERESTED IN THIS SORT OF WORK –>>THERE WE GO.>>Victor: GO AHEAD. ALL OF THE CODE I USED FOR THIS IS ALL AVAILABLE ONLINE. THERE’S A BLOG POST. THANK YOU.>>Zak: THANK YOU VERY MUCH. [ APPLAUSE ]>>Zak: THANKS VERY MUCH, VICTOR. NEXT UP WE HAVE WISDOM. COME ON UP, WISDOM. THANK YOU. THERE YOU GO. THANK YOU.>>Wisdom: HI. I’M GLAD TO BE HERE. I’M WISDOM. I’M FROM TOGO. I GREW UP THERE. AND I’M CURRENTLY A VISITING RESEARCHER IN MONTREAL, DOING RESEARCH IN NATURAL LANGUAGE UNDERSTANDING UNDER THE SUPERVISION OF (?). SO, SINCE THE PAST YEAR I’VE BEEN INTERESTED IN MEDICAL REPORT GENERATION. AND SO BASICALLY, WHEN YOU GO TO SEE A RADIOLOGIST YOU GET YOUR CHEST AND X-RAY TAKEN. THE RADIOLOGIST TAKES A FRACTION OF SECONDS TO PRODUCE A REPORT THAT HAS MOSTLY SECTIONS OF FINDINGS AND IMPRESSIONS, AND OBSERVATIONS OF DIFFERENT REGIONS OF THE CHEST, SAYING IF THERE’S AN ABNORMALITY OR NOT. THE IMPRESSION SECTION HIGHLIGHTS THE FINDINGS. RADIOLOGISTS CAN COMMIT MISTAKES. THE AI COMMUNITY HAS BEEN THINKING OF WAYS TO AUGMENT THEM WITH AI CAPACITY TO PROVIDE A THIRD EYE. AND WE HAVE STUFF FOR THAT. THE PROBLEM WITH THIS CLASSIFICATION IS YOU ARE GOING FROM THE IMAGE TO THE LABELS. WHERE IS THE STEP WHERE WE GENERATE A RADIOLOGY REPORT? AND YOU WOULD LIKE TO TRY SOMETHING TO GENERATE THE REPORTS, BASICALLY CONDITION THE LANGUAGE MODEL ON THE INPUT IMAGE AND MAXIMIZE THE LIKELIHOOD. BUT THIS DOESN’T WORK VERY WELL ON MEDICAL REPORTS, BECAUSE THERE IS NOTHING IN THIS FORMULATION THAT ENSURES CLINICAL ACCURACY OF THE REPORT. AND THIS IS A BIG PROBLEM. AND THIS IS WHAT I’VE BEEN INTERESTED TO SOLVE. AND I FOUND INSPIRATION. SO IN THIS SETTING YOU HAVE A MODEL THAT RECEIVES NATURAL LANGUAGE INSTRUCTIONS TO ACHIEVE A TASK. AND TO CORRECTLY ACHIEVE THE TASK, THE MODEL NEEDS GOOD NATURAL LANGUAGE INSTRUCTIONS. I SAID WE CAN DO THE SAME THING FOR MEDICAL REPORT GENERATION. ON TOP OF MAXIMIZING THE LIKELIHOOD, WHICH WE DO IN IMAGE CAPTURING, WE CAN ALSO REWARD THE LANGUAGE BASED ON HOW WELL ITS OUTPUT WAS USEFUL FOR THE MEDICAL TASK, LET’S SAY CLASSIFICATION. SO HERE THE CLASSIFIER TAKES THE RADIOLOGY REPORT AND WE CAN ALSO ADD AN IMAGE FOR SUPERIOR ACCURACY. BUT WHAT IS INTERESTING HERE IS IN THE BACKWARD PASS, WE ARE UPDATING THE LANGUAGE MODEL PARAMETERS BASED ON HOW WELL THE OUTPUT WAS USEFUL FOR CLASSIFICATION. THAT’S A GOOD STARTING POINT FOR ACCURACY BECAUSE WE ARE FORCING LANGUAGE MODELS TO OUTPUT THINGS THAT HAVE ENOUGH PERTINENT MEDICAL CLUES TO INFER ACCURATE DIAGNOSIS. AND I TRAINED THIS MODEL ON A DATA SET WHICH IS THE LARGEST TO DATE WITH RADIOGRAPHS AND REPORTS. AND TO TRAIN THIS, I NEEDED EXTENSIVE AMOUNT OF COMPUTE. I STARTED THIS PROJECT AS A MASTER’S STUDENT IN INDIA, ON MY LAPTOP. THAT WAS PAINFUL. SO, I APPLIED TO TFRC TO HAVE ACCESS TO TPUs AND I GOT IT. SO I HAD SUDDENLY MANY TPUs FOR FREE TO DO A LOT OF EXPERIMENT AT THE SAME TIME. AND THAT WAS USEFUL TO ITERATE FAST IN MY RESEARCH, BECAUSE I NEEDED TO REPRODUCE THE BASELINES AS WELL AS OPTIMIZE MY APPROACH. SO THE TFRC WAS 8,000 PLUS POWERS. TI USED V2 DEVICES AND GOT TO TY THE V2 POD DEVICES. SO I COULDN’T LEAVE YOU WITHOUT SOME RESULTS OF THE MODEL. SO, THIS IS A CASE OF HERNIA. AND IF YOU SPEAK — YOU TALK TO A RADIOLOGIST THEY WOULD TELL YOU THE MAIN EVIDENCE FOR IT IS THE CARDIAC OPACITY YOU CAN OBSERVE WITH THAT GREEN ARROW. AND IN THE RED BOX YOU SEE THE RADIOLOGY REPORT OF THIS X-RAY. SO THE GREEN ONE IS A BASELINE I REPRODUCED AND OPTIMIZED THE BEST I COULD. BUT THE MODEL COMPLETELY MISSES OUT ON THE KEY FINDINGS. AS A CONSEQUENCE, BECAUSE YOU ARE OPTIMIZING — MAXIMIZING THE CONFIDENCE OF THE MODEL, IT WILL AVOID TAKING RISK AND WILL TELL YOU 80% OF THE TIME THAT YOUR X-RAY IS FINE. SO THE BLUE BOX IS MY APPROACH. AND YOU CAN SEE THAT BY FORCING THE LANGUAGE MODEL TO UPLOAD THINGS I USE FOR CLASSIFICATION, THE MODEL FINDS THE RIGHT WORDS TO USE TO JUSTIFY THIS CASE. SO I’M VERY EXCITED TO HAVE PRESENTED THIS WORK VERY RECENTLY AT STANFORD, AT THE FRONTIER OF AI, SCIENTIFIC SYMPOSIUM. AND I’M EXCITED FOR WHAT’S NEXT WITH THIS PROJECT. AND I’M THANKFUL TO THE TFRC TEAM FOR PROVIDING THE VOICES THAT WE USED FOR THIS WORK. THANKS FOR HAVING ME. [ APPLAUSE ]>>Zak: THANK YOU VERY MUCH, WISDOM. THAT’S GREAT. NEXT UP, WE HAVE JADE.>>Jade: HI, I’M JADE ABBOT. I’M FROM SOUTH AFRICA SO I’VE COME A VERY LONG WAY TO BE HERE. I’M NOT ACTUALLY HERE TODAY TO TALK ABOUT WHAT I DO AT WORK, WHICH THESE DAYS IS A LOT OF BERT STUFF. MY SIDE PROJECT, RESEARCH A HOBBY. WHAT WE’RE TRYING TO DO IS WORK ON — THERE’S A LOT OF AFRICAN LANGUAGES AND VERY LITTLE RESEARCH. WHAT WE DID IS DEVELOPED FIVE OF THE SOUTHERN AFRICAN LANGUAGES, BASELINE MODELS. THIS WORK FEEDS INTO A GREATER PROJECT, WHICH MEANS WE BUILD TOGETHER IN ZULU. WE’RE TRYING TO CHANGE THE FOOTPRINT ON THE CONTINENT. SO THE PROBLEM, WE HAVE OVER 2,000 LANGUAGES, WHICH IS QUITE INSANE. MANY OF THESE LANGUAGES ARE EXCEPTIONALLY COMPLEX. IN CONTRAST, WE’VE GOT ALMOST NO DATA. IF WE DO HAVE DATA, WE HAVE TO DIG TO FIND IT. AND WHAT’S EVEN WORSE IS THERE’S NO RESEARCH. IF YOU’RE A BEGINNER MLP PRACTITIONER CURRENTLY LEARNING ABOUT MACHINE TRANSLATION ON THE CONTINUE TENT AND YOU’RE TRYING TO FIND SOMETHING IN YOUR LANGUAGE THERE’S NOTHING. YOU CAN LOOK IN SOME OBSCURE JOURNALS AND FIND OLD PUBLICATIONS AND THAT’S THE EXTENT OF IT. THIS MAKES IT HARD IF YOU’RE TRYING TO BUILD ON MODELS AND SPUR THIS RESEARCH. IF YOU LOOK AT THIS GRAPH, YOU CAN SEE WHAT IS THE NORMALIZED PAPER COUNT BY COUNTRY AT THE 2018MLP CONFERENCES. THE MORE ORANGE, THE MORE PAPERS WE’VE GOT. YOU SEE THERE’S GLARINGLY EMPTY CONTENT IN THE MIDDLE THERE. AND EVEN AT WIDENING MLP WORKSHOP, THE GRAPH STILL DOESN’T LOOK THAT MUCH DIFFERENT DIFFERENT. THAT’S MEANT TO BE INCLUSIVE OF MORE PEOPLE FROM AROUND THE WORLD. WHAT DID WE DO? I LIKE TO SAY WE TOOK SOME EXISTING DATA THAT WE FOUND AND TOOK THE STATE-OF-THE-ART MODEL AND SMASHED THEM TOGETHER. THEY’D NEVER SEEN EACH OTHER. AND WE THEN DECIDED TO TRY TO OPTIMIZE THE ALGORITHMS, PARAMETER-WISE. OUR GOAL IS TO SPUR ADDITIONAL RESEARCH. PROVIDE BASELINES. THIS IS WHERE TFRC CAME IN. THIS IS MY SIDE PROJECT. INSTEAD OF HAVING LOTS OF MONEY, I TRIED AND WAS RENTING GPUs FROM A CLOUD PROVIDER. THEY WERE COSTING ME AN ARM AND A LEG. I REACHED OUT TO TFRC AND THEY WERE SUPER HAPPY TO LEND US THESE TPUs. WE USED THE TENSOR TO TENSOR FRAMEWORK TO TRAIN UP THESE MODELS. WE USED GOVERNMENT DATA, PARALLEL CORBUSES WE MANAGED TO FIND THERE. ONE OF THE THINGS WE FOUND THAT WAS SIMULTANEOUSLY PRESENTED AT ACL ON A DIFFERENT LANGUAGE, ENGLISH TO GERMAN, WE FOUND OPTIMIZING THE CODING ALLOWS THESE VERY COMPLEX LANGUAGES TO HANDLE THEIR FLUTENATION. YOU BUILD UP NEW WORDS BY ADDING MORE WORDS ON SWITCHING LITTLE BITS COMPLETELY CHANGES THE MEANING. OPTIMIZING THIS PARAMETER CAN MAKE SIGNIFICANT DIFFERENCES IN THE SCORE. AND WHAT WAS ALSO GREAT IS WE NEEDED TO SUBMIT SOMETHING IN TWO OR THREE WEEKS TO A WORKSHOP. AND INSTEAD OF TAKING DAYS, IT WOULD TAKE A COUPLE OF HOURS TO BUILD THESE MODELS. THANK YOU TO TFRC FOR THAT. RESULTS OVERVIEW, THE FIVE LANGUAGES WE HAD. YOU CAN SEE WE HAVE ALMOST — PARTICULARLY IN ONE CASE — ALMOST DOUBLE THE BLUE SCORE. BIGGER IS BETTER WITH BLUE. IN AFRIKAANS, ARCHITECTURE, IT WORKED BETTER THERE. THAT RUNS IN CPU AND TAKES A LOT LONGER THAN WE WANTED IT TO. AND ZULU IS A COMPLEX LANGUAGE BUT WE HAD VERY FEW SENTENCES AND IT WAS MESSY. THE STATISTICAL TRANSLATION PERFORMED SLIGHTLY BETTER. IF YOU LOOK AT THIS VISUALIZATION, IT’S ESTABLISH TO AFRIKAANS. YOU CAN SEE THE ATTENTION CAPTURED SOME OF THE LANGUAGE STRUCTURES. HERE WE’VE GOT A PARTICULAR INSTANCE WHERE CANNOT IN AFRIKAANS BECOMES TWO WORDS. AT THE END YOU HAVE TO SAY NI AGAIN, A DOUBLE NEGATIVE. I’M NOT SURE WHY IT’S CALLED THAT. YOU CAN SEE THAT IT SORT OF DOES THAT LESS SO ON THIS SCREEN FOR SOME REASON. YOU CAN PICK UP CANNOT TO NI AND NI. THOSE ARE THE WORDS. HERE WE’VE GOT ONE OF OUR SAMPLE TRANSLATIONS, A SOURCE SENTENCE, THE REFERENCE TRANSLATION. VERY FEW OF YOU ARE LIKELY TO SPEAK THIS IN THE AUDIENCE. SO WE’VE GOT A NATIVE SPEAKER TO ACTUALLY TRANSLATE IT BACK TO ENGLISH, WHAT THE TRANSFORMER GENERATED. THEY’RE TALKING ABOUT SUNFLOWER FIELDS, LANDS, FLOWERING PERIOD. YOU CAN SEE IT’S DONE REALLY, REALLY WELL DESPITE HAVING SO LITTLE DATA. THIS IS MY CALL TO ACTION. I WORK ON A PROJECT, COLEAD OF A PROJECT. YOU CAN GO CHECK IT OUT. AND OUR IDEA IS TO CHANGE THIS MAP. SO THIS MAP IS WHAT OUR CURRENT REPRESENTATION OF RESEARCH IS ACROSS THE AFRICAN CONTINENT. THE IDEA IS TO SPUR RESEARCH. IF YOU KNOW A LANGUAGE FROM AFRICA OR EVEN IF YOU DON’T AND YOU’RE WILLING TO CONTRIBUTE TIME, OR RESOURCES, OR ADVICE TO A LOT OF JUNIOR TEAMS WHO DON’T HAVE SUPERVISORS OR PEOPLE WHO WORK IN MACHINE TRANSLATION, OR IF YOU’D LIKE TO DO A WEBINAR, DROP US A MESSAGE. AND THANK YOU VERY MUCH TO TFRC FOR HOSTING US. AND I LOOK FORWARD TO WHAT ELSE WE CAN ACTUALLY BUILD.>>Zak: THANKS SO MUCH, JADE. I’M GOING TO GET THE CLICKER. JADE. [ CHUCKLING ] THANK YOU SO MUCH. LET’S HAVE A ROUND OF APPLAUSE TO VICTOR, WISDOM, AND JADE FOR COMING TO REPRESENT THEIR RESEARCH. THANK YOU SO MUCH. [ APPLAUSE ]>>Zak: IT’S A PLEASURE TO HAVE YOU HERE. SO THERE’S ONE MORE PROJECT I WANT TO SHOW. JONATHAN WASN’T ABLE TO BE HERE IN PERSON BUT THIS IS FANTASTIC WORK. I WANTED TO SHOWCASE IT. JONATHAN AND HIS COLLEAGUES AT MIT WON THE BEST PAPER AWARD WITH THE PAPER CALLED THE LOTTERY TICKET HYPOTHESIS. THEY’RE LOOKING FOR SPARSE TRAINABLE NEURAL NETWORKS WITHIN LARGER NEURAL NETWORKS. THAT HAD NOTHING TO DO WITH TFRC, BUT IT WAS AN INTERESTING IDEA. MANY OF THE NEURAL NETWORKS WE’RE USED TO ARE VERY LARGE. ONE THING THE OPEN AI GRAPH DIDN’T SHOW IS THE NEURAL NETWORKS ARE GETTING LARGER OVER TIME. THERE’S A CORRELATION BETWEEN LARGER MODEL SIZES AND HIGHER ACCURACY, AS LONG AS YOU HAV ENOUGH TRAINING DATA. BUT IT TAKES MORE AND MORE COMPUTE POWER TO TRAIN THESE LARGER NETWORKS. JONATHAN AND HIS COLLEAGUES ASKED WHAT IF YOU COULD FIND JUST THE RIGHT SUBNETWORK IN THIS MUCH LARGER NETWORK THAT COULD PERFORM THE SAME TASK AS THE LARGER NETWORK TO THE SAME ACCURACY? THOSE NETWORKS ARE WHIMSICALLY CALLED LOTTERY TICKETS. AT iCLEAR, JONATHAN USED SMALL NETWORKS BECAUSE THAT’S WHAT HE COULD AFFORD, TO SHOW INITIAL ENCOURAGING EVIDENCE FOR THIS HYPOTHESIS. BUT THE REAL INTERESTING PART OF THIS RESEARCH, AT LEAST FROM MY PERSPECTIVE SINCE I’M INTO BIG COMPUTE, IS DOES IT WORK AT SCALE, RIGHT? AND SO TO FIND THAT OUT, JONATHAN GOT IN TOUCH WITH US HERE AT TFRC TO TRY TO SCALE UP THIS WORK. AND HE WAS KI KIND ENOUGH TO SAY FOR HIS GROUP, RESEARCH AT THIS SCALE WOULD BE IMPOSSIBLE WITHOUT TPUs. LET ME SHARE A LITTLE BIT MORE ABOUT HIS WORK AND ABOUT HIS FINDINGS. THE LOTTERY TICKET HYPOTHESIS IS RELATED TO THIS BROADER CATEGORY OF TECHNIQUES CALLED PRUNING. THERE ARE MANY APPROACHES. MOST TAKE PLACE AFTER THE NETWORKS HAVE BEEN TRAINED. YOU’VE ALREADY SPENT THIS COMPUTE TIME AND COST TO GET THE NETWORK TRAINED AND THEN YOU MODIFY THE TRAINED MODEL TO TRY AND SET WEIGHTS TO ZERO OR REDUCE THE SIZE OF THE MODEL OR DISTILL IT INTO A SMALLER MODEL. IT’S INTERESTING TO ASK COULD YOU JUST TRAIN A SMALLER NETWORK FROM THE START? COULD YOU PRUNE CONNECTIONS EARLY ON, MAYBE AT THE BEGINNING OR EARLY IN THE TRAINING PROCESS WITHOUT AFFECTING THE LEARNING TOO MUCH? THIS INITIAL PAPER SHOWED SOME VERY PROMISING RESULTS ON SMALL NETWORKS, ON SMALL DATA SETS. WITH THE TFRC CLOUD TPUs, JONATHAN TOOK THIS TO MODELS WE’RE ALL FAMILIAR WITH — RESONANT 50 TRAINED ON IMAGE NET. HE FOUND SLIGHTLY DIFFERENT BEHAVIOR, BUT WAS ABLE TO VALIDATE THE HYPOTHESIS. SO YOU CAN’T GO ALL THE WAY BACK TO THE BEGINNING. YOU CAN’T PRUNE ALL THE WAY, WITH CURRENT UNDERSTANDING. BUT YOU CAN GO ALMOST BACK TO THE FIRST EPOCH, CUT THE NETWORK DOWN AND THEN TRAIN FROM THERE WITH A MUCH SMALLER NETWORK WITHOUT ANY HARM TO ACCURACY. JONATHAN FOUND WITH RESONANT 50, YOU COULD REMOVE 80% OF THE PARAMETERS AND NOT HURT THE ACCURACY. YOU’RE TRAINING TO 90 EPOCHS OR FURTHER. THIS IS A REAL SAVINGS. THERE’S PLOTS DOWN BELOW SHOWING YOU THIS REWIND EPOCH AND SHOWING THE TEST ERROR STAYS LOW UNTIL YOU GET PAST REWIND EPOCH 4 OR 3. ONE THING I APPRECIATED WAS IN ADDITION TO CARRYING OUT THESE EXPERIMENTS AND INSPIRING OTHER RESEARCH, HE BUILT SOME INTERESTING TOOLS TO HELP MANAGE ALL THE COMPUTE POWER. WHEN YOU’RE SEEING HERE IS A GOOGLE SHEET — A SPREADSHEET THAT HE WIRED UP WITH SCRIPTS TO ORCHESTRATE HIS EXPERIMENTS. THIS WAS A FULLY DECLARATIVE SYSTEM. HE COULD ADD A ROW, THE SCRIPT WOULD KICK OFF A NEW EXPERIMENT, MONITOR THE RESULTS, BRING THEM BACK INTO THE SPREADSHEET, FLAG ERROR CONDITIONS AND THE SPREADSHEET HAD THOUSANDS OF ROWS SHOWCASING THE DIFFERENT EXPERIMENTS THAT WERE SEARCHABLE, SHAREABLE AND USABLE IN ALL THE WAYS THAT A SPREADSHEET IS. THIS WAS A GREAT MIX OF OLD TECHNOLOGY AND NEW. THIS WAS A SERIOUS AMOUNT OF COMPUTE. JONATHAN ESTIMATS HE USED AT LEAST 40,000 HOURS OF CLOUD TPU COMPUTE. I HOPE THAT UNDERSCORES WE’RE SERIOUS ABOUT PROVIDING A LARGE AMOUNT OF COMPUTE FOR YOU TO DO THINGS THAT YOU COULDN’T DO OTHERWISE AND SHARE THEM WITH THE RESEARCH COMMUNITY. SO, IT’S NOT JUST ABOUT PROJECTS YOU’VE HEARD TODAY. THESE ARE SAMPLES OF THOUSANDS OF RESEARCHERS WORKING ON TFRC. I’D LIKE TO ENCOURAGE YOU TO THINK ABOUT YOUR NEXT RESEARCH PROJECT HAPPENING ON TFRC. IT CAN BE AN ACADEMIC PROJECT OR A SIDE PROJECT. IT CAN BE AN ART PROJECT. AS LONG AS IT’S INTENDED TO BENEFIT THE COMMUNITY, AS LONG AS YOU’RE GOING TO SHARE YOUR WORK WITH OTHERS, MAKE THEM OPEN AND HELP ACCELERATE PROGRESS IN THE FIELD, WE’D LOVE TO HEAR FROM YOU. SO, IF YOU’RE INTERESTED IN GETTING STARTED RIGHT NOW, YOU CAN VISIT THIS LINK BELOW AND ENTER CODE TF WORLD. THAT WILL GO STRAIGHT TO US AND THE ORGANIZERS OF THIS EVENT. WE’RE HAPPY TO MAKE AVAILABLE AS A STARTING POINT FIVE REGULAR CLOUD TPUS AND 20 PREEMPTABLE ONES FOR SEVERAL MONTHS FOR FREE. THE REST OF GOOGLE CLOUD SERVICES STILL COST MONEY. BUT THE TPUs ARE THE OVERWHELMING MAJORITY OF THE COST. WE HOPE THIS ENABLES THINGS YOU COULDN’T DO OTHERWISE. IF YOU GET TO THE LIMITS OF WHAT YOU CAN DO WITH THIS, PLEASE REACH OUT. LET US KNOW. TELL US ABOUT WHAT YOU’RE DOING. WE CAN’T PROMISE ANYTHING BUT WE’LL DO OUR BEST TO HELP WITH MORE — MAYBE EVEN A LOT MORE COMPUTE CAPACITY, INCLUDING THE ACCESS TO PODS THAT I MENTIONED EARLIER. SO, THANKS AGAIN TO ALL OF YOU FOR BEING HERE TODAY. THANKS TO OUR SPEAKERS FOR REPRESENTING THEIR WORK IN PERSON. WE’LL ALL BE HAPPY TO HANG OUT HEREAFTER WARDS AND ANSWER ANY OF THE QUESTIONS YOU MIGHT HAVE. PLEASE RATE THIS SESSION IN THE APP AND THANK YOU ALL VERY MUCH. HOPE YOU’RE ENJOYING TF WORLD. [ APPLAUSE ]

One Comment

Add a Comment

Your email address will not be published. Required fields are marked *