| """ | |
| Filter Texas executed inmates by whether any of their last words fit in a | |
| list of words commonly associated with religion. | |
| A quick demonstration of the overall patterns in web-scraping, including | |
| using a HTML parser to navigate the DOM and the use of Regex for | |
| hand-entered values. Does none of the file-caching/management that you should | |
| be doing for such a task | |
| """ |
wrangled.csvExample usage:
| def print_(*args, **kwargs): | |
| """The new-style print function from py3k.""" | |
| fp = kwargs.pop("file", sys.stdout) | |
| if fp is None: | |
| return | |
| def write(data): | |
| if not isinstance(data, basestring): | |
| data = str(data) | |
| fp.write(data) | |
| want_unicode = False |
This was tested on MacOS 10.14.5 on 2019-07-16
schemacrawler is a free and open-source database schema discovery and comprehension tool. It can be invoked from the command-line to produce, using GraphViz, images/pdfs from a SQLite (or other database type) file. It can be used from the command-line to generate schema diagrams like these:
Just a quickie test in Python 3 (using Requests) to see if Google Cloud Vision can be used to effectively OCR a scanned data table and preserve its structure, in the way that products such as ABBYY FineReader can OCR an image and provide Excel-ready output.
The short answer: No. While Cloud Vision provides bounding polygon coordinates in its output, it doesn't provide it at the word or region level, which would be needed to then calculate the data delimiters.
On the other hand, the OCR quality is pretty good, if you just need to identify text anywhere in an image, without regards to its physical coordinates. I've included two examples:
####### 1. A low-resolution photo of road signs
Graphic via State of Florida CFO Vendor Payment Search (flair.myfloridacfo.com)
This is a quick command I use to snapshot webpages that have a fun image I want to keep for my own collection of WTFViz. Why not just right-click and save the image? Oftentimes, the webpage in which the image is embedded contains necessary context, such as captions and links to important documentation just incase you forget what exactly that fun graphic was trying to explain.
This combines the instructions on a few different tutorials:
The Guardian offers an API as deep and robust as the New York Times Article API when it comes to content analysis.
The Guardian's API offers more than "1.7 million pieces of content", with published items as far back as 1999. You can register as a developer here, which gets you 5,000 API hits a day and an API key that looks something like this:
zzzyyyyy-9a9z-999z-z999-9e8a83922516
The Guardian has a handy interactive explorer to interactively tweak the query parameters.
AWS Textract is now out of closed beta. You can read the features page here, and you can also read about its limits here (e.g. no handwriting). Basically, if you've ever had to deal with the hell of getting structured data out of a PDF (scanned image or not), Textract is aiming for your business:
This short gist contains some of my brief observations about Textract and its demo, as well as direct links to the most relevant and important files, such as the Textract demo sample image and the resulting data files from Textract's API. If you have an AWS account, I h
Turn this cute YouTube cat video into a briefer-but-still-cute GIF:
