The first question we had was about debugging and logging and how to make the development process a little bit smoother and any advice you had about that. Ok. Yeah. That’s a good question. Ok. So, testing and debugging. Let’s start with debugging first cause this one I actually do know how to do. First thing you want to do is in your file. You can take import logging. And this is the Python standard logging library, and it’s already configured so throughout your code, you can basically say logging debug and just give this a string, and it will print that in the console whenever this line is running if you’re in the bug mode. Now, there is another series of lines that I’m not going to write here, but if you look at the homework exams that I put up basically at the top of all my source files I have this line that’s basically DEBUG, and basically what it does is it looks at the environment to see if I’m running an app engine debug mode or in production. In production, this would be false, and in development, this value would be true. And this basically triggers all of these statements. In production, we don’t log all these debug statements, but in development we log–everything that gets called by .debug gets printed to the console. Now you could also say .error, logging.error, and this will print in production and in development, and so this is really handy. If there is a case where something went wrong, the database did not load right or some user error happened that you want to be aware of, even if it’s happening in production, you can call logging dot error, and then you can go into the admin console, which, in app engine, you go to appengine.com and you can see all your admin stuff. You can actually look at all of your logs, and it’ll include everything from error, but it won’t include the stuff from debug. That’s really helpful kind of way of seeing what’s going on in your program. Of course, it’s always print. It is really helpful as well. Now, using things like the actual Python debugger, is really hard with web apps because you’re not running this module. This is whole framework around it. There are more complicated setups, but I found over the years that just having a really good output to your console, you can go a long way. Now, for how to test–testing is really, really hard on web apps. There’s a whole Udacity course on how to do testing. And a lot of your modules, a lot of the stuff that I would put in a utils file like the hashing and the cookies and salting and all that stuff, you can just write functions to test that–there’s a whole course. Now, for actually testing the behavior of your web app that’s a lot harder. What we used to do at Reddit is we would deploy our code on one server, and we’d have an error log that was constantly scrolling in real time of all of our errors across all of the app servers, and if that starts scrolling faster, we know there’s a bug we just pushed into production, and we take that machine out of rotation and investigate. That’s I would say not the most professional way of doing things. However, I have on good authority that that many of the largest websites from the world actually do things that way because testing website is fairly non-deterministic. You just don’t know what the errors are going to be. It was very hard to know what the errors are going to be. Another strategy we used in Reddit a lot is we would take before doing a big deployment, we would capture the last 10,000 requests or so, and we would store those, and we’d re-played them against our new app servers or in development. We just ran all of those requests against the development app server. If that produced an error, then we’d go investigate. That works really well, because we figured if the error is serious, we’re going to see it immediately. If it’s not something we see immediately, it’s probably not serious. That’s basically how we’ve done our testing. Every once in a while something bad sneaks through. Fortunately, we’ve never lost data or really corrupted any data, because we try to do the best we can locally, but it’s still a real challenge. And every website is a little different, so it’s something you need to think pretty hard about. What can go wrong and what happens if this piece of data is null or is full of malicious text. As long as you kind of think through that way, you can catch the most errors while you’re developing. I hope that helps–it’s an inexact science I’ve met but that’s the way we’ve gotten by in a pretty large website.