Getting the Full Monty from Python

Author: Colton Rodgers, Infrastructure Engineer

Java usually reigns supreme throughout the coding world in the eyes of most. Though just because a language is the most used does not mean that it’s well liked or even the best tool for the job. I’d like to focus on Python’s versatility and ability to solve nearly any problem. It’s a very clear and powerful object-oriented programming language that compares to Perl or Ruby. It can also be compared to Groovy in the sense of being dynamic languages and also show commonalities in syntax. Which is the ability to “extend an application at runtime to change behaviors, types and objects.” Now that we’ve covered some ground about what Python is, let’s take a look at some of the frequent, and not so frequent use cases.

Frequently Seen Use Cases:

Natural Language Processing

Throughout my years of working with Python, the one use case that I most often come across is its Natural Language Processing abilities. The Python library NLTK allows Python programs to work with human language data. Let’s look at my development of a Sentiment Analysis Python Program that would stream in posts from Twitter. These posts contained certain keywords or hashtags, as many Twitter posts do. The program broke up the tweet’s json file, which consisted of numerous data points, such as coordinates, user interactions, language, the tweet text, and the user. It then extracted the tweet text and analyzed the overall sentiment of tweet. The program then; appended that value to the json and sent it on through the pipeline to its resting place in a Hadoop Distributed File System via Apache NiFi. Both of which are systems I plan to cover in the coming months.

Data Analysis

Python is home to some of the most powerful Data Science libraries that programmers have readily available. Namely, the MINS libraries of [M]atPlotLib, [I]Python, [N]umPy, and [S]ciPy. In combination, those tools are a whirlwind of power with abilities like data visualization, interactive computing & development, linear algebra, multi-dimensional arrays and probability distributions. I’ve had the pleasure of working with IPython through Jupyter in relation to my aforementioned Twitter Sentiment program. Jupyter with IPython creates a very user friendly graphical interface that allows one to quickly visualize their data in unison with parallel computing and a notebook interface. In the case of the twitter sentiment streams, I used IPython to view and predict the overall sentiment, the change in sentiment and the speed at which it changed and when. When you see these changes correlate fairly strongly to outside events, you can more confidently predict that certain events will have specific outcomes. This information can be very powerful if it is refined to a high level of confidence.

Less Conventional Use Cases:

Network Penetration Testing

On a whole different wavelength, Python can be used for Network Penetration testing. Network Penetration testing is essentially testing a computer system’s network to find vulnerabilities than a malicious attacker could exploit. One of the most commonly used Python libraries for this is Scapy. Scapy is a packet manipulation program that is able to forge or decode packets from a wide number of different protocols. It can easily be used for a lot of the classic tests such as scanning, traceroutes, probing and network discovery. One of my favorite network discovery tools, nmap, can be almost completely replaced with Scapy. It also adopts a lot of the similar functionality of hping, arpspoof, arp-sk, arping, tcpdump, tethereal, and other libraries of comparable utility. The library is definitely worth a look if you and your company are looking to accelerate into the area of penetration testing. There is quite a bit of documentation surrounding how to get started and some ideas about what to begin with.

Music Composition? Yes, Music Composition.

Python is one of the most versatile languages out there, so it should be no surprise to you that you can use Python to compose your own musical scores. By using the Python library ‘LilyPond,’ a programmer can create their own musical composition while disregarding the layout of the sheet and allowing them to focus more purely on the music itself. Creating a simple sheet could look similar to the following:

c’ e’ g’ e’

After compilation of that file the output of a PDF would look like the below:




Pretty neat right? I myself dabble in guitar, drums, and a little in piano and have tried this out on a few occasions and I can say that it’s a pretty nifty tool.

Whatever it is you are looking to accomplish, Python may be able to help! I frequently use it for automating daily tasks in order to free myself up for more mentally demanding projects. It can be useful in Language Processing as well as Data Science and Manipulation, but also can be used for less conventional means. Set forth to find out what you can gain from Python!