Techathon Series: Data Scraping & Structuring

At Sogody, we believe that everything starts with our people. That’s why we put so much energy into building a culture of continuous professional development and improvement, not only in the services we provide for our clients but also in the work we put in for our own team.

We are constantly looking for ways to improve our team and, as a result, improve the quality of the work we deliver for our clients. We have created opportunities for everyone to get involved and play an active role as part of our growth story.

Hence, we started the Technical Brilliance Initiative (BriTech) as a way to stay at the forefront of innovation in technological design and production.

“BriTech is a joint initiative to stay ahead, learn, and help each other grow with Sogody’s internal and external knowledge. Being flexible, learning at scope, and better understanding our trade brings forth opportunities that go far beyond our day-to-day tasks and responsibilities.”

BriTech is a product of passion and admiration towards learning. Our team wanted to turn this passion into collective growth and cooperation.

The BriTech initiative provides our engineering and design teams the opportunity to participate in new ways to solve difficult problems that can help us improve our products, our services, and most importantly, the end-user experience.

In this line of thinking, we have organized Techathon as a way to explore new tech and brainstorm new ideas. Techathons are a series of presentations, live coding, and creating.

Here is what went down in the first Techathon Series: Data Scraping & Structuring.

Data Scraping & Structuring

For the first Techathon, we had Dardan Bakiu, Software Engineer @Sogody, talk about data scraping & structuring. This was a two-part Techathon. Part one was a presentation that covered data scraping, in which the team discussed how to read data from various sources and how to build tools using different languages and libraries.

Bakiu delved into important topics about data, with an emphasis on best practices:

• What is data scraping and how is it used;

• When and why use it;

• Types of data scraping.

Our first step was to familiarize everyone with the process of data scraping. We want all team members to understand the possibilities of data scraping and emphasize its usefulness.

All hands on deck for part two!

After the presentation, our team had a live coding session where they worked on creating a tool that could be used for data scraping.

There are various ways to scrape data; however, for this Techathon, our team focused on web scraping. The idea was to create a tool that is able to properly scrape data from various resources and get it into an easy-to-use format for further exploring the data and how it is displayed.

The team built a tool that uses the Cheerio library for scraping, which then gets executed in Node.js. By using this tool our team was able to extract information from websites by selecting what information they are looking for and then the code automatically extracted those data and saved them in a NoSQL MongoDB.

The same data was then displayed in a headless UI that our team created using ReactJS.

Takeaways from the first Techathon

Having a rich company culture is extremely important to us. Techathons will be a company-wide event where employees exchange knowledge, experience, and ideas, and fuel each other’s growth — this builds strong teams that are ready to innovate.

BriTech is not just an initiative and an opportunity to learn new skills, but it also fosters a sense of community that lasts even after an event has ended.

We wanted to share with you our excitement and appreciation for the work that our team has been doing. One cannot describe the incredible feeling of accomplishment we feel when we see our team working together, not just on a project, but on something bigger than ourselves.