SparkIP

Summary:

SparkIP is an API which interfaces with Apache Spark. SparkIP was designed to streamline the analysis of massive network logs by leveraging the hugely scalable Spark framework and our new IP address data type. One of the main jobs of a security analyst is to comb over millions of network logs to identify threats in their network. The ability to efficiently query such logs not only saves time, but also makes the network safer from malicious agents. Security analysts can quickly pinpoint threats and act accordingly. Previously, querying network logs in Spark was cumbersome because users had to create complex regular expressions to match IP addresses. With our IP address type, IP addresses are baked into Spark dataframes. Security analysts can now query for specific IP addresses and IP networks as if they were querying an integer or string. Furthermore, we have implemented two IP address data structures: IP sets and IP networks. These data structures enable users to group together both individual addresses and ranges of addresses. Queries can then be run against these groups. SparkIP is currently supported for both Scala and Python.

System Illustration:

Poster for SparkIP

Team Members:

Ryan Jacobs

Bio:

Ryan was born and raised in Los Angeles, but made his way up to Northern California for university. He graduated in March of 2021 with a BS in Computer Science. He is interested in computer networks and backend programming. Recently, he has experimented with frontend programming using React and Typescript.

Email: rybread42@gmail.com

LinkedIn: https://github.com/RyanMatthewJacobs

Github: https://www.linkedin.com/in/ryanmatthewjacobs/

Prateek Narayanan

Bio: Prateek Narayanan is graduating in June 2021 with a BS in Computer Science. He is interested in full-stack development and will be starting a Master’s in Software Engineering at RIT in August.

Email: prateeknarayanan@gmail.com

LinkedIn: https://www.linkedin.com/in/prateeknarayanan/

Github: github.com/prateeknarayanan

Julian Shalaby

Bio:

Julian is a fourth year Computer Science student who is graduating this Spring. He has a passion for software engineering, which he will pursue after graduation.

Email: jshalaby@ucsc.edu

LinkedIn: https://www.linkedin.com/in/julian-shalaby/

Github: https://github.com/jshalaby510

SparkIP

Other Partners Day 2021 Projects