Now examine the source code at github/cisc474

A place to play with backend hosting Cloud 9 IDE

• HTML5 tags: cheatsheet, and more official documentation.
• CSS/JQuery Selectors: cheatsheet, official CSS docs, official jQuery selector docs.
• DOM events via jQuery documentation.
• IDEOne can give you a python terminal if you want to hack at the class list
• Sample JSFiddle randomization code, including jQuery and bootstrap
• Building a simple unbiased shuffler
• The project: Here is a list of the 6 discrete math groups for MATH210. In jsFiddle build a randomizer which picks 3 groups to present to the TA and 3 to turn in a written solution. Then give the TA the ability to (at her whim) pick a random speaker from a group. The lecture starts at 12:20 so the best solution will be used live today. Email me a link to your program so I have a record of your work.

  Here is a link to the syllabus version of the project specs

  Email me your finished product at yourname@js.prof.ninja Use title Project 1: Your Name, Your Quality Score

  Array version of names

  An HTML canvas driven heart animation (The source code is viewable as 'heart.js')

  JSFiddle which button example

I want you to take two classes (and a weekend) to build a boardgame selection app, first the backend, then the frontend.

About the app: I have a large collection of boardgames. Boardgamegeek provides a collection tool (also an XML based API) which can export someone's collection. (When you are done with my version you could probably build apps for other power users or the community at large.)

I would like to get a random list of games to play. It would be nice to filter this list by the number of players currently at the table, the heaviness of the game desired (each game is rated by users on a scale of 1-5 for heaviness), the rating of the game (each game is rated 1-10 for quality), or other interesting filters that you think up.

Today we will set up a backend. I'm providing a database via github (also raw here at web.prof.ninja). Also a simple demo version is live at cloud9 (source code here)

In this case you will practice reading from an Sqlite3 database, handling GET requests and URL Parameters with PHP, making AJAX calls using jQuery, rendering results using HTML, and designing a basic read-only API.

Your API should return JSON data as a response to valid requests, you can pick which requests you will support and what the interface will be. Here are some sample API-style calls you might want to handle:

• GET /games
• GET /games?maxplayers=3
• GET /games?minweight=2
• GET /bestranked?recommendedwith=2
• /getrandom.php?number=5 (That is, if simple enough you could just have a bunch of php links rather than a single file API)

When you have designed your contracts, put them into a documentation file (like a README.md). When your backend works, email me a link to some api hook showing me something interesting from the database

• Sqlite3 Documentation
• PHP PDO documentation
• PHP Superglobal Documentation ($_GET, $_REQUEST, etc)
• jQuery ajax documentation
• Documenation on HTTP Status code meanings
• RESTful HTTP Verbs (with kittens)
• SQLITE3 Aggregate Functions (avg, count, sum)
• On a mac you can install XAMPP and run you own localhost with apache, mysql, and PHP (but you will probably lose some hours on setup and tuning the server)
• Likewise you could sign up for the github education pack and then sign up fo digital ocean hosting

• If you are still unsure of how to proceed here are some steps:
• Pick a url, let's say http://mysite.com/game_app/api/hello.php
• Find an api directory or make one using a command like: mkdir api
• Make a file named hello.php
• Make it return a JSON object of the form {"hello":"world"} when you navigate to the above url. Something like echo "stuffhere" maybe json_encode could help.
• Now that you are getting comfortable, connect to the db (copy my code for now) using PHP PDO
• Return all of the game names using a query like select * from games
• Think about the ways of mixing and matching the data. For instance explore the anatomy of an SQL query
• Now that you know SQL (no pressure though), figure out how to make an API
• Make it the goal to have index.php as your one file (for now) and have it do hello world when you hit the url http://blah.com/game_app/api/hello now without the .php.
• Now that you've mastered making APIs, learn to handle parameters
• That is take the above url (I'll shorten, as is common, to /hello) and adjust it to /hello?key=value
• In your PHP file see if you can return {"key":"value"} (and change them to see)
• Now design your API
• When all of that is ready we can explore AJAX (next class)

A few thoughts for students that might be very uncomfortable so far:

1. I expect that you feel uncomfortable right now. That is because I am giving you things to do BEFORE you know how to do them. I believe you have already learned more by these struggles than you would in 3 weeks of a traditional lecture format. If you disagree then let me know.

2. If you feel lost now that does NOT mean that you will feel even more lost as the course goes on. In fact I think you will feel more and more comfortable because of the struggles.

3. My plan is this: cover the basics of building a stack, any stack, first. Then settle in on making it better and analyzing your alternatives.

4. By having a project you will have a clear goal. Technical skills will stand between you and the goal. Learning to spot the missing skill and conquer it is a skill more valuable than any specific skill.

5. Thoughts on grades... You all gave me a set of desires at the beginning of the course. I would like to see you learn the things you desired to learn (and a basic level of all skills). At the end of each project ask yourself this: did I earn some skill points in one or more topics with this project. If not then push yourself to do more. If so, then feel good and try to push yourself in the areas you wanted to learn most. I want my grades to reflect your personal growth NOT the overall quality of your projects. If you already know what is going on then pick something you do not know and work that way. We all have to push ourselves to develop skills.

6. Discomfort and Uncertainty... Theses are two fears which people tend to avoid. I believe that learning to face discomfort and uncertainty will have a positive impact on your life. To move from what you currently know to what you do not know requires leaving your comfort zone.

The Github repo and Cloud9 versions have had several updates.

• My dumb API now returns more than just the game names, observe how.
• The HTML/JS Front-end (which does not have much function yet, that is your job) is now functional in three different ways.
• The jQuery approach now has clickable games that will alert some data
• There is an angularjs approach using some basic data-binding.
• There is a basic MVC in Backbonejs approach.
• All three do roughly the same thing (allow clicking a specific game)
• This is the beginning of Object-Oriented Javascript and the beginning of MVC web development

Here is Andy's quickie version of basic web hosting and the ethos behind a PHP driven API:

1. Everything should be reachable using a URL (universal resource location)

2. A specific url in your browser can be broken down into "protocol://subdomain.domain.top_level_domain/path?url_parameter=value". As an example, when I view this page in my browser I see: "http://js.prof.ninja/notes" which would mean that I am using the http protocol (hyper-text transfer protocol) the top level domain "ninja" the domain "prof" and the subdomain "js". The domain name typically refers to all three of the domain parts: e.g. "js.prof.ninja".

3. When you have a web host your "domain name" will be pointed to a directory which will have files. When a browser (or whatever) makes a request to a URL the "server" will try to find a file from within that directory to respond with/send to the requester. For instance in the folder you are currently viewing I have a file named "andy.html" which tells you a simple truth, if you visit http://js.prof.ninja/notes/andy.html then my server will check to see if that file exists and if so it will send the file down to your browser which will render it (using the HTML standards). If you try to visit a file which doesn't exist then you will get a 404 error (or some other 4** error depending on the configuration of the server). For instance try accessing nofilehere.html.

4. A web server is just a normal computer which has a constantly listening program that wants to handle HTTP requests. There are many programs which can run on a computer waiting for requests. Apache is the most common program which waits for requests and serves up files. NGINX is another popular server, and Node can act as a server using its module: Express.

5. Whenever you have a public web server there will be a folder which is "hosted" at a given domain name. This site has a folder named "public" which has all of the files I want to be available to the internet at large. Often the hosted folder is referred to as "/". Inside the folder "public" is a folder named "notes" (the folder you are currently looking at). Everytime you add to the "path" at the end of a domain name you will be heading deeper and deeper into the nested folders. For instance "/notes/folder1/folder2" will be resolved by looking inside of "public" for a folder named "notes" and inside of that for a folder named "folder1" and inside of that for a folder named "folder2".

6. When you navigate to a folder which exists you will see an index of that folder's contents. Try it in a folder I created which has a subfolder and two files, folder1. This is the default behavior for viewing a folder. If however I were to put a file named index.html or index.php then the server would show you the "index" file rather than the folder's contents. This is a setting of the server, so "index.txt" would not be displayed for a folder.

7. You can use this trick to display a file without having to use a file extension. For instance if I wanted the url for my blog to be "js.prof.ninja/blog" but my blog was a big long file I could make a folder named "blog" with a file inside of it named "index.html". Now my grandma would only need to remember the url "js.prof.ninja/blog" rather than "js.prof.ninja/blog/index.html"

8. PHP is programming language which can be used to run programs on the server BEFORE serving the file to the client. The reason I chose PHP and SQLite for this project is because both can accomplish all of your goals with basic files. (While a solution using node, rails, flask, or other server-side frameworks would require getting a process to always be running on some server.) Here is a PHP file which will tell you the time at the moment it was served to you. That file looks like this:

   
                   <?php
                       date_default_timezone_set("America/New_York");
                       echo "The time is " . date("h:i:sa");
                   ?>
                   

9. The idea behind our "API". An API (application programming interface) literally only refers to a contract that a program will honor (call my code in this way and I will get you data in that way, etc). In the world of web tech and API has come to refer to RESTful data services. A lot of people will argue in forums about what is REST and what is not REST but I don't want you to worry too much about that for now. Here is a thought experiment for you, imagine I want to do a request to /notes/andy/29 and get back the year when I turned 29 (2012), but /notes/andy/42 should be the year I will turn 42 (2025). Using the trick I showed you above you could make a folder named "andy" and inside of that put a ton of folders named "1", "2", ... , "29", "30", ..., "42", "43", ... etc. Each of those folders could have a file in them named "index.php" and they could each be programmed to return the appropriate year.

10. Sounds silly right? If I want to have a "variable" URL which doesn't worry about files but is used to allow requests from web apps where I might not know all of the values then we need a little help. This is where the ".htaccess" file comes in. It is in the github repo for the boardgameproject inside the "api" folder. ".htaccess" is a file which Apache looks for to know what to do in a given file. The one I give you re-routes ALL file requests in the "api" folder to the "index.php" file. Now I can do a request like "api/andy/40" and the request will serve the "index.php" file inside of the "api" folder. That is much better than a 404 error.

11. Once you have a file like "index.php" that will handle ALL requests into the "api" folder you can break down the URL to figure out what was requested. Now the power is in your hands to do what you want for specific requests.

SQL Now that you can respond to requests however you would like you just have to learn to play with databases. For this project we can use PHP PDO to access the database I provided. SQL is not a particular database just a "Query Language" that guides how to query databases, insert into databases, update them, etc. For this project we are only worried about "reading" from the database. The basic command looks like this select something from table where condition. "something" is the column or columns you want results for, the "table" is the collection of data, and "condition" is a way of filtering all of the rows in the table. For instance select objectname, rank, average from games where minplayers > 2 and maxplayers < 3 would grab the name, rank, and quality of everygame which can be played by 2 or 3 players (but no more than 3).

Thoughts on the MV* revolution Javascript frameworks have been all the rage in the last 2-3 years. If you would like to explore the many options please visit ToDoMVC, the 99bottles of javascript frameworks. More thoughts later.

This should be our last day of board_game_app work. You guys are becoming more comfortable with the backend and now need to make some basic AJAX calls. I added tons of comments to the javascript files, index.html, index.php, and fetch_desc_bs4.py files in the github repo (and on cloud9). This is so you can teach yourself by reading the code more effectively than reverse engineering my thoughts (thank Bola for the suggestion). We have much to discuss today.

I want to discuss

• AJAX requests and 3rd party services (cross domain request issues)
• JOIN and LIKE as SQL commands
• Basics of Object Oriented JS
• Basics of Angular and Backbone
• Our new TA Sohail Khan
• My proposals for our next project

New app time. The votes are in and we are going to build a "chat-room" app. In the last app I was very light with my specifications and many students were confused by how their project would be graded. I want to make it clear that the goal in this class is ALWAYS to make a product that an end-user would enjoy. We are pretending to be start-up CEOs. The second goal is to learn skills that you want to learn. I will be happiest when you are proud of your work.

For this Chat-Room app I want some sort of a lobby and some sort of rooms. It could be that the rooms are two player tic-tac-toe games and they can chat while playing, or that people can make rooms about their favorite topics and go there to discuss.

Think up something interesting that would need real-time interaction with the server. You could build a choose your own adventure chat, or a meta language to have the server give you weather data or friendly advice. It could be a collaborative "Eat poop you cat" server (look it up), or a Taboo game where one person gets a word and they have to get the other people to say it... Let your imagination go nuts.

The technologies involved:

• NodeJS and SocketIO

  Node is a server-side javascript engine which is good at handling many users at once. SocketIO is the de facto websockets library for Node, web sockets are a two way connection to the server (rather than AJAX you send/receive messages). Both have tons of literature on the web. I have built a hello world app (github version) (cloud9 source version) for you to explore today. Also cloud 9 has a working chat-app as their default node app, so make a new space running node and you can play with their chat app.

• Client side javascript (again)

  Another type of question that is common when you get started in learning javascript is how to get data from users. I built a very simple sandbox for reading user inputs in various ways (using jQuery). Angular will have its own methods which I might hack together over the weekend.

• Single Page App

  This is another buzzword out there now that forms a new paradigm in web development. In the old way you click links to other pages and those pages are served along with their javascript, css, and assets. The new paradigms are all about minimizing bandwidth and saving server-trips (except perhaps this websockets thing). Single Page Apps are traditional pages which use javascript (and HTML5's new history.pushState functionality) to pretend to be many pages. This works by having actual history logs that work with your forward and back buttons to give the illusion of travelling to many pages. The benefit is a single setup cost. You can get all of your libraries loaded and assets lined up and have a traditional web surfing feel without needing to reload resources at every new page.

  I have provided a "vanilla" single page app which is not for the faint-hearted. This was me trying to prove to myself how Backbone and Angular get it done. Most folks use a javascript framework to introduce single page app routing. I will say more about this next Tuesday.

• Third-Party Proxies

  I mentioned this outloud but I wanted to you show you a more live method of using third-party services. At least two projects lined up for this class will require using a third party service and probably more. So I set up another sample app to see how you could be a CORS-friendly proxy. It also runs on cloud9 with source here. The app gets stock quotes from Yahoo's YQL service. If you learn YQL there are TONS of amazing things you can pull off.

The chat app project does not require any database (per say). We are stretching our legs with three new technologies. NodeJs, SocketIO, and Single-Page-Apps. I encourage you to also begin thinking about Object-Oriented Javascript and the notion of Model-View-Controller. Since the last class I have built many simple apps to demonstrate some concepts Can't miss this... top seven new apps made by Andy for you:

• Single Page App written in Backbone source on GitHub running live at js.prof.ninja/bb_spa
• Single Page Angular App, GitHub Source running at js.prof.ninja/ng_spa
• Andy's scrabble training game using node and socketio (source) running (sometimes) at Cloud9
• A jQuery example of basic Classes, Objects, and basic MVC running at jsFiddle
• A Ninja directive made in Angular (in the next generation of Angular directives will replace controllers) running at Plunker (superior to jsFiddle I'm afraid)
• Multi-Room Chat App on GitHub running at Multi-Room Chat App on c9.io
• STOOPID SIMPLE multi-page "app" on github and on Plunker

There should be lots of comments scattered throughout that code, although it's probably one month of the industry average for code produced so I might have missed some explanation. If there is something I need to explain just let me know.

Reference links: There is a lot to learn here so I should give some references that might help in your learning:

• SocketIO Documentation
• Backbone Documentation
• Angular Directive Documentation
• Node Documentation
• Express Documentation

Other useful links: In prepping for this course I stumble onto many things I want to share but don't fit into any particular topic. So I'll toss some links in now:

• Command to kill apache on cloud9
• Angular in 60ish Minutes also as a YouTube video
• Differences to come in Angular 2.0 (youtube)
• A nice exploration of Cross-Domain AJAX issues
• Nice Multi-Room/Channel socketio advice
• Great advice about real world programming as business (a strong case for refactoring code)
• Some drunken circles on jsfiddle

Still a lot to learn from the previous examples. However I do want to give you two more gifts:

• A live demo of taking a project you like and creating a GitHub Repo from it.
• The repo I create will be an AJAX playground for making many types of requests and handling them in various ways.
• Live AJAX playground now on GitHub too

So I'll show the same things I wanted to show (some basic git repo work). I also built a Backbone Demo "Email" App to play with event based programming. Here it is on plunker (no backend). Here it is on GitHub. The chat app due date is pushed back to next Tuesday (March 17th).

An informal guide for creating a github repo from an existing folder

Light CSS play. Side-project today. I want you to take the CSS Zen Gardens base HTML and make it pretty. Use your own stylings, make it art. Make it at codepen.io or some alternative but use only CSS (I would allow pre-processors like Stylus or SASS). Turn it in using the submission guidelines on March 24th. Extra credit if you do a cool design for js.prof.ninja too.

• A ready to go CSS Zen Gardens raw setup at CodePen.
• A more fleshed out example, demo of basic tricks
• A wonderful palette picking tool at Paletton.
• Some great concept learning tutorial/guides.
• Mozilla's CSS learning resources
• Object Oriented CSS
• Preprocessors (Stylus)
• Twitter Bootstrap
• Think about CSS resets
• Google Fonts
• Flex box docs
• CSS Avengers
• Non stop flow of designs

Also this is really cool:

See the Pen Canvas Snake Bugs by Jack Rugile (@jackrugile) on CodePen.

Created a MEAN-stack upvoter (live at upvote.prof.ninja) in order to help us make some choices for the upcoming two weeks.

Code is on github, it uses MongoDB (a binary JSON, NoSQL database) and that took me some work to set up. Here are some cloud9 instructions.

We haven't yet done a database design project, so I wanted to introduce mongo sooner rather than later. You have to think differently about your databases when you go NoSQL. Another contenter as a NoSQL database is couchDB.

SQL is the established way of thinking about databases. You won't really go wrong learning MySQL or PostGreSQL, they are fast and prevalent. But they are rigid (hard to change your schema) and difficult to scale (as your app grows what sort of problems will you face).

NoSQL is sleek and scalable using JSON-esque structure, flexible data storage, and elegant searching. In my production applications I've choosen MySQL, but the size of my clients was small and certain to stay small. The choice impacts the rest of your architecture so it's good to know the options.

Next choices (vote now)

• MEAN stack reddit clone
• HTML Canvas driven app
• Responsive Mobile Deployment (PhoneGap)
• Authentication options
• Module-based Growable Project (do it one way, revamp the specs, keep most of the code)
• Move on to small team mode, solving problems for small businesses

Ask an artist session A friend of ours that is a talented artist with a good technical eye (BFA from FSU, worked at Smithsonian in art restoration, studied the field since youth) has agreed to answer questions about graphic design. If you want to put some questions up for her at upvote.prof.ninja then we can take the best and she'll give us answers and maybe a short Skype call at some point in the near future.

Small Business projects We have 7 (or so) potential "clients" that will help us go through the engineering and specification clarification stages of a live project. They all have needs, and we have talents and the desire to serve. We probably don't have time for each team doing each of the 7 projects. So we can do "sprints" where each team picks a project and does that project until the next deadline, or limit the number of projects.

NEXT PROJECT Due April 7th: MEAN stack Reddit clone

Use a MEAN stack (Mongo, Express, Angular, Node) to build an upvoter system with authentication (oAuth using a third-party).

Three gifts today:

1. Simple Static Authentication Example live at cloud 9. The method is using apache htpasswd. It doesn't have much dynamic action going on, but it is a quick way to protect developer dashboards.
2. OAuth example using GitHub and Node this example uses passportJS which has about 140 different authentication strategies. It is live at cloud 9.
3. Additional comments in the mean upvoter source code which was built by loosely following this tutorial

Rambling Authentication thoughts: I didn't give an example app for storing your own passwords and usernames. I've done this many times myself but it is probably not the best practice going forward. I might make another one showing how to anyway, but not yet. Here's why: running your own local authentication is likely to be a weak point security wise. You'll want to hash the passwords on the client side and use an SSL certificate to make sure that no sensitive data is sent or stored in the clear (including the hashed version of the password).

Best practice is to only store hashed passwords in your database and rehash new passwords to see if they match. If someone forgets their password then allow them to reset it. That way you don't know their password, in case your server is compromised.

Since this can go so terribly wrong I only recommend doing it in very small projects. The oAuth strategy is to use a third party that both your client and you trust (like Google, Facebook, Twitter, GitHub, LinkedIn). They probably have their ducks in a row. You ask your user to get a token from the trusted third party which they give to you to use to get information about you from them. Github's OAuth explanation is pleasant to read and worth the time.

In either case your app will end up with something that identifies the user of your product, a username, UUID, or email. Unless you are doing a very simple app in which all you need is their name (which our reddit remakes could be that simple) you'll want to have the user information in a database. In my systems I've used "roles" for each user, and then the various parts of API are open to certain roles only. This is now in the realm of software engineering, our class textbook has some wonderful insights.

Typically the sensitive user data is available to your app through session data (think cookies). You could also pass around a token in the headers of requests if you don't want to lean on cookies (I've never messed with it but I can see the merits). For now see if you can have someone login using a third party and you display their name, try Google+ or Facebook or Twitter.

There are many wonderful tutorials out there, this one caught my eye. But no matter what, this is going to cost you some hours (setup of the DB, creating a login screen, creating the landing, creating the backend handling of all of this). As we move on as developers we are going to want to keep a lot of this functionality in boilerplate projects that we can re-use. Almost every real life project will require authentication, so investing in a decent code base is worth it if you want to make more than just a few apps.

1. From here on out we can work in groups. Retake the skill survey, if there is a particular skill you want to specialize in keep that in mind while moving into group mode. If you are in a team with other people that also want to specialize in your area then you might want to switch around.
2. Group Selection Time. If you have a self-selected group of 4 already then move on to the next step. If you have 0, 1, or 2 partners then find them and decide the following: On a scale of 1 - 10 how industrious are you (and your partners), AND On a scale of 1 - 10 how confident are you (and your partners) as web technologists? I'll guide the rest.
3. Each team should pick 2 projects from the projects page and email me your team members and which projects you plan to tackle and in what order.
4. I plan to look at each student's reports so far over the break, your level ups, hours, and give some specific feedback.

Moving on up to your own hosting

As we move into larger projects (from cloud9, mostly for using more elaborate databases than just SQLite) I recommend getting your own web-hosting account. Usually this is $5-$10 dollars a month. You can also get your own domain name (there are many cool top level domains now (like ninja)) for $5-$10 per year. As mentioned earlier in class there are many viable options: Digital Ocean has a free $100 discount as part of the GitHub Education Pack (although there is a backlog). Amazon web services has EC2 setups for free (but they are complicated to get working). I personally recommend webfaction, in your group of 4 you can split the cost and make it $2-3 per person per month of class. My friend likes A Small Orange (but I think it is less developer friendly). If you are a do it yourselfer then most of our computers can themselves act as a host but you'll have to hack at your local router to allow random traffic in (a security risk for sure). Again I recommend webfaction.

To get a cool domain name hooked up (not strictly needed, just cool) you'll buy the domain name from a registrar like namecheap or godaddy (don't buy the funky any last minute services, just the name). They will have a way for you to update the DNS records. This is a cool topic which you should read over at a high level. Update the nameservers to point to your webhost's name servers. Here's a walkthrough of the process for webfaction. In general you can then set name records, A records mask a domain name to a given IP address, CNAME records send a domain name to another url, and MX records make email addresses resolve correctly.

Typically the host will let you know which domain names point to which folders. Those folders are now hosted publicly just like on webfaction (or plunker for that matter). I like to use subdomains to point to many different folders on the same hosted machine. Here is webfaction's guide to getting a website set up.

The MEAN stack broken down into mini-goals

Also as you move into larger projects they can start to seem too big. You want to think about how to break the project up into modular chunks which are each manageable. In your teams of four this becomes very important and it is a great meta-skill to work on. Here is a sample breakdown for our MEAN stack reddit-clone project.

• Get a node server up and running and saying "hello world"
• Using express, get a node server to act like an API, that is, a GET request to "/topic" should return a dummy-array like ["list","of","topics","here"].
• Separately get a MongoDB process running. (the webfaction mongo install guide)
• Be able to see a mongo shell and type help
• Pick a basic tutorial to work through
• Connect your node server to the mongod process.
• Try running a cronjob, which is a process that can run at regular intervals. See if you can add a line to a file every minute (then stop after a few).
• See if you can make a shell command to start a process (node or mongo) if it is stopped or ignore the process if it is running.
• Run that command every 20 minutes using a cronjob (alternatively explore the node module "forever") for instance the following two lines keep my upvoter running:

                  
      19,39,59 * * * * ~/webapps/upvote/bin/start
      15,30,45 * * * * if ! pgrep mongod; then \
      /home/novocin/webapps/upvote_mongo/mongodb-linux-x86_64-3.0.0/bin/mongod\
      --auth --dbpath /home/novocin/webapps/upvote_mongo/data/\
      --port 26329 --fork --logpath /home/novocin/logs/user/mongod.log;\
      fi;
                  

• Now see if you can call the "GET /topics" request from a real URL
• Now see if you can have "GET /topics" pull data from the database
• Now see if a "POST /topics" can be made to create new data
• Now see if a "PUT /topics/:some_topic_id_here" can update a topic, somehow
• Now have node also serve a client folder
• Get to doing the non-setup design work.
• Feel free to find a tutorial, or switch to PHP and Sqlite (LAMP), if you give up on the MEAN stack.
• Bug your new team-mates for their insights (be sure to elevate the level of your group and not to encourage desperation, this is a difficult task at first)

First, there are at least 5 students that were added late as fifth group members. They should find each other now and become their own group.

Today's Gifts

In honor of the beginning of group work I want to share some books that I have found helpful in progressing from small-scale to full-scale projects.

1. The pragmatic programmer: from journeyman to master A very helpful set of guidelines for improving your craftsmanship, each well defended
2. Growing Object Oriented Software Guided By Tests
3. Clean Code: A Handbook of Agile Software Craftmanship
4. Patterns of Enterprise Application Architecture (our textbook!)

Bowling addendums. I talked with the owner, and he clarified that all of the fractions which are needed in the project have denominators 2, 4, 8, 16, 32, and 64 (drill bit sizes I think).

When starting big projects I like to mimick the brainstorming that IDEO promotes (the deep dive) (this link is to the first part of a three part series in which they walk through the design of a "new" shopping cart).

The things that have stuck with me:

• First gather data, learn anything you can about the area and share.
• Second have a zero criticism, fixed-length creativity session in which no ideas are rejected in anyway.
• Third collate those crazy ideas into a few best notions.
• Fourth turn the winners into prototypes (maybe on paper, maybe with jsfiddle, maybe wireframed)
• Fifth pick the winning notion, either from the existing or as an amalgam.

A vaguely agile approach: Gather the specifications as "user-stories". A user-story is something which someone should be able to do, e.g. "a user should be able to login using their twitter account". (Sometimes it helps to use index cards for the user-stories.) Pick an easiest user-story and call that 1 story-point. From a baseline of how long you think it would take you to do that story estimate the story-points for the rest of the user-stories. Now try coding some number of user-stories in a fixed amount of time. Afterward you will know how many points/day you are knocking out and you can estimate if you need to cut features to finish on time.

A Yes/No Survey I worked on a side project for BHIC the Blue Hen Investment Club (options trading tutorials start tonight at 4pm in the Lerner trading lab (with the cool ticker)) which used some of our tech. So I'll share that with you now. Code on GitHub and running live at bhic.prof.ninja.

Sample task breakdown: In the README.md I laid out the rough specification I gave myself to break the project down into digestible chunks. After struggling with cookies and sockets at the same time I settled on a browser specific UUID which I store using localStorage.

UUIDs: Universally Unique Identifiers are useful tools for web technologists as they allow you to generate keys that identify objects that will not accidently have the same value. I encourage you to check them out.

LocalStorage: is another important nice tool to know about when developing web apps. It is a method of storing key value pairs on a client's device in case you get disconnected, they refresh, or leave and come back at some point. The storage is limited to a couple of MBs but it can help when you have a very quick project that you don't want to build a back-end for. Some specifications can be found here and a demo here (view main.js)

Free Front-End Development Mini-Course: A five week seminar starting Friday the 10th 6:30pm to 8:30pm in the Venture Development Center. Details on facebook.

Additional Bowling Data I've added an email from the owner to the projects site

Git Insights: So as you work in groups I encourage you to share code using git. You can use your status in the cisc474 github organization to create repos that you all can share with. If you want to push some code to github please refer to my previous notes on the matter. Here are some of my thoughts on how to use git.

The most basic usage of git is as a time-capsule for your code. Git works inside of a particular folder (by having a hidden .git folder in that directory. When you want to keep track of a particular file type git add your_file_name_here or even git add *.txt to get all text files of git add *.* etc. Then whenever you make a change run git commit (do this often). Personally I like to type git commit -am 'my commit message here' so that I need to involve some text editor. If you get in the habit of commiting often then you won't live in fear of breaking your code. You can always rollback to the last working moment. When I end up needing to rollback there are a couple of ways to do so, typing git log will bring up a list of your previous commits. You can checkout one of them by typing git checkout uuid where uuid is the random-looking string associated with each commit in the log.

Git branches Another nice feature of git is the ability to make branches. You begin in the master branch and can create new branches at any time (or really from any point in the past too). git checkout -b my_new_branch then commits can be cherry picked to be reintroduced into the master branch.

Git Team Workflow So connecting a git repository to a remote server that you host (like webfaction or even a github repo) will let you send the latest version of your code out into the world. You connect to a remote repo by doing a git remote add nick_name url command (url and nick_name are to be replaced with correct values). You can work in several different modes. Now you can do git push nick_name branch_name (typically git push origin master) and git pull nick_name. Your team can all push and pull from the same repo which might create some problems, or you can each have your own repo and push and pull from each other (less problems). There are many many posts and blogs out there about this, here is a diagram I like:

Testing APIs with non-GET requests If you are building an API and want to see if a GET request to /surveys/123 returns a JSON object with data about survey number 123 then you can go directly to the url http://whatever.com/surveys/123 and see the JSON. If you want to see if a PUT request updates the survey then I recommend using something like the chrome extension POSTMAN to ping the API directly using various verbs.

Data Decisions

At this point we have built front-end interfaces, we have built server-side APIs, accessed SQLite from PHP, accessed Mongo from Node. Most of our projects are full-stack in the sense that they encompass everything from the UI to the data storage layer.

Database Design Choices When building a data-storage layer you will probably be going with SQL, NoSQL, or Plain-Text. If you have only a single user and very little hope of growth a plain-text file might be all you need, in which case JSON, XML, or even CSV will do the job. As soon as you have more than one user interacting with the data at the same time you'll want to use something more.

SQLite does the job well enough for small projects, it is a single file, which takes out most of the sting in setting up a database. I have heard various reports about the practical cutoff from SQLite to something more advanced but in my experience 4-10 simultaneous users on a bad shared-host was enough to keep a medium sized SQLite database frequently corrupted (if you are having corruption issues let me know I'll share some tips).

PostgreSQL is the next level up in complexity although I haven't worked with it before. Their tagline is the "world's most advanced open source database". Their competition is MySQL with the tagline "The world's most popular open source database". Both of these databases run a dedicated process which you connect to (much like your Mongo experience). They run faster and more reliably than SQLite (IMHO) and will take you up to the next user-level.

If you go with any SQL based data-storage solution then you will have to design your tables well. The rule I always remember for table design is Rely on the key, the whole key, and nothing but the key. These are the quicky versions of the three normalization forms. The goals of database normalization are that you never have repeated data (so that updates don't need to coordinate all of the possible locations of the data).

Another consideration in your database design is your use of the index. Indexed columns will hash the values in those columns for closer to instant lookups of the required rows. Too many indices and creating new data can take longer and longer but the right mix is required for good performance. Great performance training for SQL databases (use the index luke).

NoSQL databases have much less structure and typically worse performance. There are two advantages in the NoSQL options (Mongo, Couch) 1) Early in your design process you can revamp your schema often. 2) They are made with cloud scaling in mind (sharding doesn't just refer to an over-ambitious toot). Cloud-based architectures work well for "static" content (media, html, js files) but for databases writing and distributing the data does not make much sense in the cloud. You can (poorly) mimick a NoSQL database by having a "json" column in an SQL database that you keep miscellaneous data in.

Data Access Layer I want to make a case for a Data-Access layer. This is a part of your code which is controlled by the API and interacts with the database. In small projects (one-off projects) it is not really needed, but if you need to start to scale a project then you'll want to think about how you might go from one database provider to another without too much rewriting your code. On a smaller scale even adding or adjusting your schema should be doable without fear that things will break.

For SQL-based storage there is a natural tension between making tables which mimick the objects of your domain (classes that make sense to your clients and/or code) and normalized efficient tables. For this problem I offer two solutions: 1) VIEWS and 2) ORMs. Views are virtual tables built from SQL queries. The advantage of views is when you want to have a table whose columns match the attributes you want but that aren't proper for database design you create a view. ORM stands for Object-Relation-Mapping, this is a design-paradigm in which certain objects understand how to access SQL (relational) data. There are many nice ORM libraries, for PHP, Node, Ruby, Python, and Java. I have had luck with ADODB in my PHP driven APIs.

By using an ORM for data-access you can, in theory, limit the impact of upgrading your database as your needs grow. Also your code will get shorter and cleaner as your data access can be blended with your objects and the SQL parts can get hidden away. Of course every library added has some overhead.

Other fun javascript stuff

Three.js a 3d-library for making awesome things

Physijs a physics plugin for threejs

WebGL fundamentals webGL provides lower level access to visualizations through graphics cards (with graceful degrading)

Javascript style guide (best practices) a wonderful set up guidelines used by many great companies (I fail on many of these)

BeepJS A synthesizer library for javascript

Free Color Flat Icons

I built squirrel-wars with Source code on GitHub. Some of the technologies that you might want to notice: Time-based code setInterval and clearInterval for repeating a function over and over. I use this to make a "ticker". I also use the pub/sub pattern (or maybe it's more of an observer pattern) to let items hook into the ticker or jump out of the ticker. Documentation on setInterval, the one-time version is setTimeout.

My intention was to demonstrate the right way to do angular components, where new objects would know how to render themselves. After some thought I decided that I was trying to make Angular into Backbone. In Backbone I might wrap a model with a view and inject that where I want it. In angular the HTML drives the show, ng-repeat is probably the way to go. Now it doesn't mean that I can't have objects that know how to render themselves but it's a bit more akward, here is a decent directive with templates.

Today I wanted to talk about mobile development. The reason javascript is so cool right now is that we are doing what was envisioned for Java. That is, a site we develop will be available, immediately, to everyone using the internet. I don't require someone to have anything else but a sensible browser (not IE8 or below). So given that the appeal of web-development is ubiquitousness we must be prepared for our content to be viewable on a plethora of possible devices.

There are two directions to take this thought. ONE: making websites that are mobile-friendly. TWO: make web apps from your javascript codebase.

Mobile-Friendly Design: The two major differences between viewing a site on a desktop and phone are 1) screen width and 2) touch-based UI.

Responsive Design: this is a buzz-word these days for good reason. The traditional field of webdesigners (up until 2012 or so) worked from photoshop files to pixel-perfect websites. They did all sorts of things that we look at as old-fashioned. A brief history of web-design. Some of the more interesting things were Flash for everything (the infinite is possible at zombocom) and tables for layout (both no-no's today). One of the stop-gaps (which I recommend against) is having a mobile version of your site, or different sites for many different devices.

Today it makes more sense to think about your site in a fluid way. That is, imagine it at many different screen resolutions and design it to look good in all of them (not just some short list of them either). Do this using the same HTML content. This concept is what responsive design is all about.

Now, when you are out google-searching for responsive design info I want you to be aware that there is a huge set of web-designers that are actually not very inclined to learn javascript. This means that much of what you read will talk about design concepts and not the actual logistics of what really happens. As high-level CS students I think you'll want to know the how and then you can make your design choices from that.

Basically there are two methods of implementing responsive design (at the moment (media queries are a hack)): Media Queries and Javascript.

Before I show you how media queries work I want to make you aware of an important relationship in the web world. There are specifications for the web which are abstract and desired. These specifications are implemented by browser makers, but some specifications are ignored and others altered (and IE6-8 just get them mostly wrong). The body politic of specification aware users make drafts and requests for future specifications which slowly work their way into browsers. So if you want to see where the web is heading or you want to shape where the web is heading or you want to implement browsers then you'll want to be aware of the specifications. I invite you to poke around at the w3c (world wide web consortium) which maintains the recommendations.

Media Queries The CSS specifications allow blocks of CSS rules to apply only in certain situations. The intention of this was that you could have better accesibility if you adjust your displays for different media types. The specifications recognize braille, embossed, handheld, print, projection, screen, speech, tty, and tv. They also allow you to insert future media types (like 3D) which will get ignored. So Web-Designers abuse this a bit, the devices with browsers are all "screen" type. But they figured out that you can use some logic in the media query and get your rules to apply in certain cases. Example code that colors the backgrounds of all h3 tags salmon when a screen is 100 to 500 pixels wide:

            
            @media (min-width : 100px) and (max-width : 500px) {
              h3 {
                background-color : salmon;
              }
            }
            
            

So in this way you can have different CSS code apply in different situations. Here is a sample to play with on codepen with three "breakpoints". Now this is how things are done using only CSS until the specifications advance a little bit farther (which it always does). This media query is particularly useful for going into "print" mode. You can have your site go into a more appropriately colored, less image intensive style for printing a site by using @media print. Sites styles in this way have "breakpoints" where one set of CSS rules disappear and the next start to apply. This annoys me when it is obvious.

Twitter Bootstrap Bootstrap is a quick way to make a raw site look better, but also to natively become responsive. Now during my tenure as a web developer I've seen bootstrap go through many iterations, as soon as I put down specifications it will date this page (which is fine by me). Bootstrap (as well as other "grid" frameworks) creates a 12-column layout within any div. You can let an element take up as many of the twelve columns as you would like. The responsive part of bootstrap lets you specify the number of columns you take up at different screen widths. <div class="col-lg-4">one third across on large screens</div> A simple bootstrap "responsive" column. You can do some amazing things with the various grid libraries out there (like responsive Fibonacci squares), I like singularity for SASS and jeet for stylus.

Javascript for responsiveness you all are talented javascripters by now. So you can always react to screen size using javascript and it will always be more powerful than CSS alone. A codepen showing javascript reacting to screen-size. However please try to use the spirit of the CSS Zen Gardens, in which as much of the style as possible is purely CSS. Use javascript as a last resort for responsive stylings, and probably you should just use javascript to adjust classes and let CSS react to those classes being changed. I had one project which used a pane-layout and bootstrap would not react properly to the panes being resized. So I put together a little javascript to detect pane size and adjust the BS classes there. This sort of thing is useful but shouldn't be your first solution.

Mobile UI Now just because you are responsive doesn't mean your site will work on a mobile device. Anything that relies on hovering needs to be rethought for a touch screen! Sometimes this can be tweaked, sometimes you just have to think differently. I encourage you now to be aware that a touch screen has things going on which are different than on a webpage.

Publishing web-apps as native-apps when you want to make a mobile-app you can choose to develop natively on the platform which will run your app or use javascript! Not everything will work as well but libraries like PhoneGap and steroids allow you to just code everything as a web-app and then compile a native application. These are called Hybrid-Apps and besides some light performance issues they can take you pretty far as a mobile-developer. The main advantages are that you can keep one code base and push both to Apple and Android without having to learn swift, objective-C, or Java (though most of you know Java already).

I don't mind sharing with you a hello-world project I did to play with phonegap, easeljs, and the issues of touch vs mouse. This is "Moggle" a simple addition boggle. The logic code is more elaborate than the UI. Drag your mouse around and try to make chains of additions that match green stars. The longer your chains the more points. Anyhow, a few hours after having a playable version I was able to put together a native iOS version using phonegap. here is the phonegap output on our github page.

Modular Ideals

We have talked often of modular front-end code. The ideal is web-components, that is chunks of code which you can reuse in many different projects. Further I want the components to be ways to visualize my classes. One of the emerging libraries for this is ReactJS, which I don't know. So Keith (one of us) has written up some thoughts on it and shared some code with us. Thanks Keith!

Testing websites

When it comes to testing your web-apps there are several types of testing that matter. There is application-level testing, that the code is performing its appointed task well (see below TDD). There is user-testing which makes sure you are building the right interface (check out usability testing on ten cents a day from "Don't make me think"). There is behavior-driven-testing which tries to make sure you are building the right features (by having your client help you build the tests). There is performance testing which tries to keep your app speedy. Now there is even platform-testing to make sure you do the right things on various browsers.

Just recently Google has started to provide mobile testing and promised to punish non-mobile-friendly sites in their page rankings. For many companies page 2 of a google search is where you die.

For speed there are several solutions, including another Google tool for analysis. We have been working with a naturally faster setup than traditional sites (APIs send small infrequent requests), but there are many tricks yet to be learned. You can ask clients to cache your intensive assets, and even do stress tests of many users on your sites. Brian Youse will talk more about this next Tuesday.

Testing Javascript Code

I wanted to introduce some methods for application-testing code in Javascript, but first a visual tour of my impressions of Test-Driven Development (a discipline that we should probably all aspire to).

TDD (Test driven development) in pictures:

Logistics of testing JS code

In my projects I have used mocha, chai, and phantomjs which you can see in my boilerplate project. Phantom is used to simulate a browser on the server, chai for its pleasant vocabulary, and mocha for its nice testing framework. If the project is important enough (makes actual money) I want to run these tests automatically when I deploy the code. My favorite technique is the post-receive hook that I told you about weeks ago. When I'm about to deploy the code, it will first run the phantom test, and report any issues to me. The code that does this is the same as what you would run from a command line. For me my post-receive hook looks like this:

            
#!/bin/bash

GIT_WORK_TREE=~/webapps/apps git checkout -f
mocha-phantomjs --reporter dot http://estateauctionapps.com/home/#tests
            
            

I do the same for PHP-driven backend code. But there I don't need a browser simulator so I just run php on the files. Like this:

/usr/local/bin/php54 -c /home/novocin/webapps/api/public/php.ini\
-f /home/novocin/webapps/api/tests/testUserRoles.php

I really great resource that I stumbled onto years ago was a test-driven javascript assessment test (for potential employees).If you are thinking of working in javascript one day, it's nice to work through these mocha-driven tests, which all fail until you make them pass.

Angular has its own testing culture, built right in. They encourage Karma, an alternative to Phantom, which can simulate specific broswers on your server, Jasmine and excellent unit-testing language, and Protractor an end-to-end testing framework. I'm no Angular expert, so I forked an excellent looking Angular-Test Patterns github repo as a starting point on proper angular testing. My impression is that the natural dependency injection is ideal for "stubbing" and "mocking" parts of your code that aren't done or should be mimicked as far as your objects are concerned. In particular angular has an ng-mock directive which can simulate $httpBackend requests.

For projects that you want to host on GitHub (so future employers can see perhaps) there is a Continuous Integration service called Travis-CI. It will run tests you ask for everytime a commit is pushed to your github repo. You've probably seen the little green badges on the top of some readmes.

Some walkthroughs on Angular testing and TDD with Angular.

Bryan Youse was supposed to talk this morning about how to start your own web server, apache vs NGINX, mod_rewrite, mod_proxy, and some basic concurrency testing. I was supposed to have my first decent night of sleep in ages... But as of 4am he is not feeling well. Send him positive energy through your nose chakra.

Instead I will talk to you about a very important topic: file uploading (think picture taking). But first an important aside about the general DOM model.

Document Object Model (DOM)

It strikes me that we haven't really discussed some behind-the-scenes concepts which are important to webtech. This hasn't stopped us from creating some bad-ass things, but maybe it will help clear up some of the formalism under the surface of what is going on.

The Document object model or DOM is a platform-neutral model (think framework) for how a dynamic document could be built from a set of objects. All web browsers interpret your html as a document that matches the current DOM specifications. This is the web API and you can contribute suggestions for future DOM specifications. For instance now the "web applications" group is in charge of the DOM specifications.

Anyhow, every html tag you make is interpreted by your browser as an instance of the Element class. The Element class inherits from the Node class and the EventTarget class. If you really take the time to learn the entire API of Elements and the other useful parts of the DOM then you might never use another javascript library. Normally development speed keeps this from happening, but sometimes when a library feels too magical it's nice to see under the hood.

Javascript is our way of interfacing with the objects that live in our HTML documents. In particular I want to draw your attention to the standard events which specify that elements should have events triggered on them in mayn wonderful cases like: pageshow, mouseover, keydown, focus, blur, change, dragenter, drop, copy cut, click, wheel, volumechange and many more. Creating callback functions for these events is what really unlocks all of our dynamic web app creation. jQuery and Angular are quick ways to hack into a few of these, but for full power it is nice to spend some time reading these docs.

Client-Side Files

I wanted to say all of that because you are all now on your own custom paths to empowerment. So some are still jQuery addicts, some angular, some react, some vanilla and then on the backend some are node, some PHP, some flask (a python version of node). Talking about the DOM allows me to talk to everyone despite your tools of choice.

First of all, if you use an <input type="file"/> tag you will get the native file uploader dialogue. On a desktop this is a file browser dialogue box, on a device with a camera you will get the choice to take a photo. (The other options for letting a user specify a photo are: let them give the url of the photo, let them drag a photo onto an element (more later).)

When an input box has had a file added to it (or the phone has finished creating the photo) there will be a "change" event fired on it. The file is now available as inputElement.files[0] an attribute. Here inputElement is the DOM element which you could grab with $("input:file") using jQuery or document.getElementByID using javascript directly.

You can now process this data in a few ways. The specs allow for a FileReader class which will allow you to gain access to the contents of a file that has been given to your app. Or you can send the file directly to your server where it can be processed and your front-end/client-side code responds.

FileReader, here is a great tutorial on using FileReader directly in the client side, along with CodePen example code. I made a jQuery version at js.prof.ninja/smile.

Sending Files (or general FormData) to the server

So far we have played with sending URL parameters to an ever eager listening server. This data is send with a "content-type" that is typically JSON or XML or application/x-www-form-urlencoded. When it comes to sending files (possible binary files) we end up using a multipart/form-data the browser handles the difficult bits, but you can inspect the requests (using your console) and you'll see that they use a UUID string to chunk off which parts of your request have which types (binary vs plain text of some sort).

To pull this off doesn't actually require much on the client-side. Typically we use the FormData which is a Class made to wrap up key value pairs from forms. (Back in the day all client to server data was done with inputs where the name attribute was what filled out the keys.) I can create a var fd = new FormData() instance which can be filled using code like fd.append("mypic", $("input:file")[0].file[0]) then sent using normal ajax. Or use a standard form to prefill it new FormData($("#myform")[0]) before sending. As a warning if you are using $.ajax for this then set processData: false and contentType: false (see $.ajax docs).

Angular folks seem to recommend libraries to deal with this, but at least one person uses the data the same way I would.

Server-side File Handling

This depends on your setup but in PHP it is as simple as:

$_FILES["mypic"][tmp_name] which you can copy into a new place using move_uploaded_file($_FILES["file"][tmp_name], "newlocation")

Put it all together

               var thePic = $("#picIn")[0].files[0];
               if (thePic.type.match(/image.*/)){

                   var fileReader = new FileReader();
                   fileReader.onload = function(e){
                       $("#picOut").html("<img id=\"newPic\"/>");
                       $("#newPic").attr("src", fileReader.result);
                   };
                   fileReader.readAsDataURL(thePic);
                   var fd = new FormData();
                   fd.append("pic",thePic);
                   $.ajax({
                      url: 'upload.php',
                      type: 'POST',
                      data: fd,
                      contentType: false,
                      processData: false,
                      cache: false,
                      success: function(data){
                      }, 
                      error: function(data){
                          alert("error");
                      }
                   });
               } else {
                  $("#picOut").html("Please give me an image.");
               }

<?php
$origname = $_FILES["pic"][name];
$name_parts = pathinfo($origname);
if(move_uploaded_file($_FILES["pic"][tmp_name], 
    "latest.".$name_parts['extension'])){
        echo "1";
} else {
        echo "0";
}
?>

For node, I hear that (tutorial here) multer is the multipart/form-data library to beat.

Bryan Youse spoke about system admin style concerns. Including firewalls, mod_proxy, mod_rewrite, NGINX, apache, nc, and nmap.

Time is slipping

We have four classes left including this one. How would you all feel about presenting your final products either to me and Sohail or the class as a whole? You have all been doing a lot of work and should be proud of your new super-powers. I have 6 topics left on my to explore list for you all:

• HTML Drag and Drop
• GitHub Pages
• DNS records
• Using CDNs in a workflow
• Basic MySQL setups
• RSS feeds
• Infinite Scrolling
• Regular Expressions
• Beautiful Soup
• Underscore.js
• Code minification

If you have other fundamental topics that you want or need to explore let me know now.

BTW grooveshark is dead. You can access your localStorage through the javascript console if you need to get your music choices back. I'll play with Plug.DJ for a bit, if you want to help dj class you can.

Target super-tasks

Also, with two weeks left, I wanted to give you a list of super-tasks. My hope has been to encourage you to grow your own super-powers and these tasks should be doable for you now that you've been most of the way through this course. If you don't think you are capable of tasks on this list then see me, learn what you need to know, you are each capable.

• You should be able to display HTML at a url of your choosing.
• You should be able to create multiple pages that link to each other.
• You should be able to load javascript libraries into that page.
• You should be able to write your own javascript files that you link into a page.
• You should be able to style a screen with CSS and load it from a separate file.
• You should be able to make a responsive, mobile-friendly site.
• You should be able to make a single-page app that has multiple screens without needing to load fresh assets.
• You should be able to make javascript classes.
• You should be able to create MVC-style javascript.
• You should be able to bind your models to your views.
• You should be able to load multiple "views" into one screen (that is make components/modules/templates that you could populate a screen with).
• You should be able to collaborate using git repos.
• You should be able to use the input tag like a boss.
• You should be able to make AJAX requests from your client-side code.
• You should be able to unit-test your javascript classes.
• You should understand RESTful APIs
• You should be able to create and implement a REST API accepting GET, PUT, POST, and DELETE with parameters.
• You should be able to link server-side code to a database.
• You should be able to Create Read Update Destroy in the database of your choice.
• You should be able to schedule a cronjob.
• You should be able to reply to a request with JSON.
• You should be able to open a websocket connection.
• You should be able to think in an event-based paradigm.
• You should be able to do third-party authentication.
• You should be able to do your own authentication.
• It would be nice if you were able to do some web-scraping.
• It would be nice if you were able to use third-party APIs to gather data.
• It would be nice to have a portfolio of projects that you are proud of.
• It would be nice if you have a workflow that allows you to quickly begin new projects without too much effort.

GitHub Pages

When you have work you are proud of GitHub Pages are a traditional and quick way to make a pretty front end. It is a basic CMS which github hosts. Of course we can make our own, but having your work on github has some advantages. A cool username.github.io url (or your own) and seeming like part of the tapped in community. From any project you can click settings and use the automatic page generator or create a gh-pages branch if you want to make your own rather than use a template.

Drag and Drop

One of the cool things in HTML5 is drag and drop! We can make any tag into a draggable item by setting a draggable="true" attribute. We can make any tag into a drag target too, but this time by having it listen for a drop event. The source element will also get a dragend event fired on it. The magic part is that you can work with the drag events and save data which the other element will get via event.dataTransfer.setData and event.dataTransfer.getData methods. Logistically that is all. Data can be sent at start time and received at drop time.

A great tutorial here and full documentation here.

In my wacky Backbone codebase I created a drop-in set of drag and drop rules which I'll share with you here and it would get called in this code. Here is the first one found using a google search of jsFiddle simple drag and drop for a less over-engineered example.

A python script I used to send emails.

For the final session in this class, in the spirit of our just do it approach, we'll have each team present their work for ten minutes. Tell us about the best "AHA Moments" (copyright Andy Novocin 2015), features you are most proud of, skillz to pay the billz, etc. In general if an individual in your group has some cool stuff show that too, if not show the groups efforts.

Your personal reports are very important to me. That is the only thing I have to judge you on. If you want to send me some specific code snippets that show your efforts (particularly if you think that I think that you've been slacking) then do so before the semester ends.

Link to our room for music: Plug.DJ for a bit, if you want to help dj class you can.

MySQL setup

If you haven't set up a MySQL database before, it's pretty stinking simple. On Webfaction it's a one-click install, most webhosts have a similar process. If you are a command-line warrior then install it, setup a username, and start running. I would advise setting up PHPMyAdmin as a GUI. I used to do all of my DB work from the command line, but I find that using the GUI I move faster. It works like Mongo more than SQLite in that it is a server process which you start and connect to, as opposed to a file that you manipulate. Different server OSes require different methods of install, but it should be a pretty standard process.

CDNs for your workflow

So on the opposite side of the spectrum is Content Delivery Networks. You probably have noticed cdn as part of the URLs of various libraries you've been including in your apps. CDNs are the static part of the cloud. I want many copies of my fixed files in many datacenters around the world. If one goes down, no biggie, the next one will take over.

In this way your content is delivered in a robust and speedy way. So what should you have on a CDN and what should you have on a standard server? Well unless you started in with Firebase then updating your data and accessing your database is probably something for your server. Your performance will be better up until some number of concurrent users. For other processing you need to do you'll want a server doing your work. But anything static can probably be hosted on a CDN.

So most of your webapps are actually static, despite feeling so effortlessly dynamic. The html, js, and css files can all be cached (once you are ready for the public, during development this is not true). Your images and videos should be cached too. In my workflow I use Rackspace (files), this is for my antiques company which has thousands of images to load into our eBay descriptions daily by our international customers. I can't afford to have those images go down, so I use a CDN for that. Another cool option that is worth considering: cloudinary which has a cool API for URL based photo manipulation.

You can, if you are really cloud-hungry, make your entire workflow distibuted and scalable. Using something like iron.io, which runs workers for you. That is, make a process and hook into their messaging system. Static content can be served via CDN and any processing you need can be handled by a worker. Much like other things in this class, it is overkill at your current level (it is overkill at my current level) but if you find yourself with a successful startup needing to scale and scale quickly keep it in mind.

Image-Magick

In my efforts to give you a ton of references here that you might wish to expand your knowledge base with, here is one my favorite web-tools. Image-Magick is the command-line version of Photoshop. Normally the command is convert or you can use libraries to access Image-Magick classes in your PHP or Node code (or python scripts). You can do anything with Image-Magick, I have used it to build minecraft mona-lisa pixel maps which can be painted with sheep colors by robots. I have used it to automatically create montages of images, 1x1 pixel secret images, banners, animated gifs (or at least the frames), PBM data for science, convert handmade PBMs into normal pngs and whatever, etc. It is a great tool to play around with in any of your scripts. Learn it, love it!

This snippet converts image.png into output.ppm using only the colors found in pal.ppm: convert image.png -dither FloydSteinberg -compress none -remap pal.ppm output.ppm

The Super-Power Show-Off!

Sign up for a premium spot using this doodle. Grades in this class are entirely based on your projects of course. That requires looking at your growth reports, hours worked, super powers gained, and subjectively assessing your growth. So turn in all of your projects and make an impression, it can only help you.

Link to our room for music: Plug.DJ for a bit, if you want to help dj class you can.

All you need to know about DNS

So all of our computers pretend like they are servers, listening on various ports on the localhost. Your router has an IP address and can redirect requests to your computer if you allow it. Likewise the servers which host your sites have IP addresses to. You can make a request to a server using the IP address alone like http://74.125.228.197/. However domain names make it easier to remember, and allow for multiple sites on one IP address. That site is better known as google.com.

Domain names are routed to IP addresses via a network of servers around the world which do a distributed lookup. The first step in the chain is a root level request to see which servers specialize in top-level domains (TLDs are like .com, .org, .ninja, .rocks, etc). Then the servers in charge of your TLD lookup your domain name. Normally this is sent to a set of nameservers. Nameservers are now, more or less, run by your organization and are off the main highways of the internet. They resolve the subdomain to the appropriate IP address, which then resolves to the appropriate process (apache, nginx, node) like normal. I'm not lying, try it out yourself.

So after that, what do you do with the info? Well when you own a domain you get to create DNS records which tell the nameservers how to direct your lookups. The major tools in your toolbox are CNAME records, A records, and MX records. CNAME records direct subdomains to different addresses. A records direct domain names to IP addresses. MX records direct your email requests. Here are some screenshots of some example DNS records:

raw IP addresses for a container of some of my media files. estateauctionpics.com has these nameservers:

Underscore (Lodash)

So underscore is a great utility belt of useful and simple functions you always need. After starting to use underscore, coding javascript felt much more like coding Python. It inserts tons of functional things like, map, reduce, filter, and reject. It can be used to work with objects. It can be used to fight NaN. (Please treat yourself to NaNNaNNaN...NaN Watman.)

Lodash is a drop-in alternative to underscore which is there to help keep things fast in the world (much like GMP vs MPIR). Speaking of, you guys should check out jsperf (here it is testing various versions of map) this is a way to test low level javascript snippets for execution spead.

Regular Expressions

Regular expressions are a right of passage for all coders. For many they are the first true super-power you develop. It makes you feel like this:

Here is a RegEx cheatsheet to help you get started. Also checkout this artist search regex tool I built to help decipher difficult signatures of artists (try [Ss]m[ae]{1,3}th to see some Smeths).

My advice for regular expression learning is this: fire up a a python terminal, import urllib2 and import re (the native python regular expression module) and see if you can write a regular expression (use the cheatsheet) to find all links in this document. That's a doable task which will make you feel like you've added the regex skill to your toolbox. Other cool goals: find an earnings calendar site and scrape all companies which have earnings on Wednesday.

Once you've done it with regular expressions you should also investigate Beautiful Soup which is a great python HTML/XMl parser.

I sincerely hope that each of you has grown as a developer this semester. As I warned you at the beginning this is a humbling field with a fast growing set of technologies. That makes things easier and harder as you always need to stay on top of what is fresh while having a core workflow that allows you to produce fast quickly. I'll keep this set of notes up at http://js15.prof.ninja for as long as I own prof.ninja. If you need any help feel free to email me at yourname at js dot prof dot ninja.

RSS feeds

So this is a little bit on the old-school side, but it shows up all the time. RSS (really simple syndication) is just an XML spec that content producers use to spread news and content. Typically you can use RSS feeds to create dynamic content on your site without needing to produce it yourself. There is another format called ATOM which is pretty similar but more actively updated.

As an example here is an RSS item from CNN's news feed:

<item>
    <title>Iraq: ISIS No. 2 killed in airstrike </title>
    <guid isPermaLink="false">http://www.cnn.com/2015/05/13/middleeast/isis-al-afri/index.html</guid>
    <link>http://rss.cnn.com/~r/rss/cnn_topstories/~3/MVgq3wcHwfo/index.html</link>
    <description>The Iraqi military said Wednesday that ISIS' No. 2 leader has been killed in a coalition airstrike -- a claim that the Pentagon said it does not corroborate... (I deleted stuff here)</description>
    <pubDate>Wed, 13 May 2015 15:35:06 EDT</pubDate>
    <media:thumbnail height="51" width="90" url="http://i2.cdn.turner.com/cnn/dam/assets/150113191855-isis-beheading-story-top-top-tease.jpg" />
    <media:content height="51" width="90" medium="image" type="image/jpeg" url="http://i2.cdn.turner.com/cnn/dam/assets/150113191855-isis-beheading-story-top-top-tease.jpg" />
    <feedburner:origLink>http://www.cnn.com/2015/05/13/middleeast/isis-al-afri/index.html</feedburner:origLink>
</item>
            

How to use it. Now that you guys can script you can do this in a few ways. Either you publish your own content by just creating and regularly updating an RSS file. Or if you want to use RSS feeds, then have a script scrape them regularly and produce a version appropriate for your workflow. There are some tools out there to get JSON versions of RSS feeds directly like Google Feed and Yahoo Pipes. Otherwise you can do this much like our earlier 3rd-party proxying.

Infinite Scrolling

So this is a design trick that popped up in the mid 00s. The concept is to have a page of content which could go on forever but doesn't need to load an infinite amount of content. Pinterest and Facebook both use this trick.

Some methods to pull this off. Anytime someone scrolls your page a scroll event is triggered (the resize event is probably needed too). The event can trigger a callback (like all events) and then you can choose to load more content from your source. Good practice in this case is to have a boolean flag that lets you know if you are currently fetching server data so you don't end up with a ton of requests. The other thing you need to do is detect if your user has scrolled far enough down to need more content. If you time this right you can give the illusion of no loading at all.

There is also an angular library that wraps up a lot of this functionality for you. I built a simple demo for you, code on github. If you want a more raw version I can give one to you on request.

XLRD is a great python library for hacking at Excel files

Pagination is a nice paradigm for flow control from API to client

Obfuscation/minimization of your javascript.

Avoid SQL injection attacks

SSL certificates

Ionic vs Phonegap

Wireframing

User Testing a la "Don't Make Me Think"