Integrating Web Scraping into a website

[slayer202]

First off, I'm a huge noob. I know next to nothing about programming, but I'm determined to learn. I want to create a website as a hobby(I've made crap ones in the past) so I want to learn how everything works as opposed to paying someone to set me up with something I can't understand.

Right now I'm trying to learn web scraping. I'd like to be able to grab data(movie times/listings, to be precise) and manipulate them onto a webpage however I like(and I'd like users to be able to manipulate them further, through whatever options I set up).

After a bit of research I've been trying to learn how to pull the data with Python and/or BeautifulSoup. Needless to say I'm having trouble getting going, but I only started trying yesterday. My main question, though, is whether or not I should keep on this path to reach my end goal. Am I better off using another technique? I don't want to waste time learning this if there's a better way, or if there are any dead ends I'll hit later. I've come across a few other ways people do this kind of stuff, but I know little about any of them. Can they all be used with equal ease to build the pulled data into a webpage?

Thanks in advance, and apologies for my ignorance

 

[WannaFly]

Scraping is definitely one of the *least* preferred ways of doing this. If you are scraping fandango, for example, I could almost guarantee that it is against their TOS. Also, if they change their layout, you'll probably have to change your scraping code, which means you're site will (probably) be down until you've updated it.

You're best bet would be a web service, but a quick google search doesn't return much. Here's one link I found:
http://www.xmethods.net/ve2/ViewListing.po?key=uuid:CB063561-6F59-F51F-4A46-B0DDB8FDD6D1

 

[slayer202]

I did look around for some sort of API or service, but they seem limited. I'd like to avoid paying for a service if possible, at least for now

I certainly don't know much about it, but I was under the impression that scraping for "facts" like this was ok. I looked through the TOS of one site, I forget which, and I didn't see anything about it.

 

[CuriousMike]

If you use PHP, "$html=file_get_contents ($url);" loads the entire HTML into a variable, $html.


From there, it's a matter of parsing that variable and pulling out the data you want.
Regular Expressions would be a common way to search for your data.
Plain strstr's or other string manipulation methods will also do it.

 

[Aluvus]

I did look around for some sort of API or service, but they seem limited. I'd like to avoid paying for a service if possible, at least for now

I certainly don't know much about it, but I was under the impression that scraping for "facts" like this was ok. I looked through the TOS of one site, I forget which, and I didn't see anything about it.

Some sites forbid "automated access" (or some phrase like that) of their site in their terms of use.

The issue of what you can do with that information afterward is a separate question. In general, mere facts are not subject to copyright, although a specific presentation of them may be.