Skip to contents

woRldnews::News Headlines using R

Eshita Zaman, Coskun Erden, Alex Cleveringa

2022-05-09

newsAPI.Rmd

Our plan was to build a project that will help graduate students to find jobs. Specifically jobs in academia. They can look for different kinds of jobs, for postdoctoral positions, assistant professor, research positions. We planned to add various search criteria to customize the job serach. Initially, we thought about searching by zip code/area/state, job description, job starting time, salary etc. We decided to use the data from HigherEdJobs.com. When we started working on the implementation, we noticed that HiggherEdJobs.com does not have an API to access their data. We decided to changed our data source but strict to the initial project idea. We decided to use data from indeed.com. Fortunately, indeed.com have a public API for developers. One can register as a developer, get the API key and secret code to get the access token. These tokens are active for an hour. After the time is elapsed, another token needs to be generated. But then we faced a roadblock. Apparently, we cannot access indeed.com’s user data anymore. The API, secret and access token can be used to post job in theirs website. But we cannot fetch data from theirs server. We the tried to change the data source again. We tried for couple other job posting websites such as glassdoor, github jobs, stockoverflow jobs. Our attempts were not successful. Some of them discontinued their APIs and some asked to signed a contract not to share data. We appreciate the cautions big companies are taking to protect user data. Because user put sensitive information in the job application. It can be dangerous if that information goes to wrong people. We had to change the project idea because of not getting the data we were looking for. Finally, the main takeaway points for us while selecting a project were:

  • Make sure the API exists
  • Make sure it’s a public API
  • API is currently operating
  • Data is not extremely sensitive
  • API allows to fetch data from server

Our current project is to get top news headlines everyday. A person can search for today’s top news headlines based on country, language, category such as health, sports, entertainments etc. Top headlines can be serached based on news sources like BBC, CNN, Aljazeera etc. One thing to remember while searching news headlines is county and source cannot be put together in the same search.

In this package, you will get to know

  • Getting started with the package
  • Registering for the API key
  • Functions
  • Example
  • Shiny app

Shiny app

In this section, we want to introduce the interactive user interface that user can use to run the application. When you run the shiny app it will create a window in the user’s preferred browser and ask the user to select the parameters for the news search. Based on the parameters, the application will search the news headlines and display them in a tabular format. In that data table, user can see the news title, news description, published date, news url and image associated with the news. If you click on the news link, it will directly take the user to the original website where the news was published.