Trials in Snake Charming

So I managed to scrape and analyze my first chunk of html from a live website using this handy tutorial.

I installed and learned how to use the lxml and requests python libraries, and learned how to copy and use the Xpath of an html element.  lxml, combined with requests, lets you request the full html from a given url and then structure it into a data tree that you can traverse.

You can then inspect an element using developer tools built into a web browser and copy it’s Xpath. You can reference the Xpath of a particular tag class to pull any data with that given given tag from the data tree you created and put it into a handy list object.

When I tried this with my first webcomic however, I got an error when requests tried to decode the html.  Apparently dealing with html encoding from site to site is something you have to learn to deal with as a professional, and my next research topic.  After I get that sorted I should be able to write code to scrape what I want scraped, and then I can learn how to insert it into a cron job that runs the scrape automatically on a schedule and my project will be finished!

So I finally got jQuery working with the help of my mentor. Here is my first jQuery code!

$(document).ready(function(){
$(‘#id_collections’).change(function(){
window.location=this.value;
})
})

Now my dropdown menu correctly sends users to the collection view of their choice upon selection of an option. I also wrote my first working context processor so that all of my django views can universally display the dropdown without having to write the render code for it over and over again.

Next I have to get images with title and date tags to appear in the collection view, and all of my front end work is finished!

Lots of distractions over the past month and a half with my son starting school again among other things. I dove back into my project in earnest this week and solved the problem of getting my new comic request form to work on every footer of every page. I simply pointed it to a separate view in the end.

Now I need to get a drop down menu that lists collections you can view to go to a new page displaying the collection. I have it where each item on the list contains a value that tells the new view which url to use, but I need it to automatically submit url when the appropriate item is selected from the menu. My mentor has suggested that I use JQuery for this. So far I have downloaded the JQuery file and put it in a scripts directory in my static files. Next I have to figure out how to properly activate in my HTML and use the appropriate function call to make my dropdown behave how I want it to.

So I got a more advanced template structure up and running in my project thanks to Django’s inheritance feature in the template system. It is a HUGE time saver and lets you reuse redundant code like that from headers and footers with one simple django command instead of copying and pasting the code into every template every time you change it.

When you’re working with an app like Django, its really important to read the documentation and understand what shortcuts are available in the latest version.

I was looking to customize my form for creating comics, and when I looked online there are a million convoluted ways to get it done, many of which were from old versions of Django.

In the end the documentation told me it was as simple as calling the custom widgets with just one variable in the form and then calling the field names attached to them in double curly braces {{}} in the template. Bottom line, make sure you check your documentation every time you go researching and avoid a lot of headaches.

I have the final design of my landing page complete!  I haven’t written it up in html and css yet, but this pretty much exactly what it will look like.  Now I need to take care of three more layouts and my design will be complete.