Web scraping with Google Apps Script

by

Go to that website in your browser and open the developer tools (F12 or ctr-shift-i). Click on the network tab and reload the page with F5.
A list of requests will appear. At the bottom of the list you should see the asynchronous requests made to fetch the information. Those requests get the data in json form from footballapi.pulselive.com.
You can do the same thing in apps script. But you have to send a correct “origin” header line or your request gets rejected.
Here is an example.

function fetchData() {
  var url = "http://footballapi.pulselive.com/football/stats/player/2064?comps=1";
  var options = {
    "headers": {
      "Origin": "http://www.premierleague.com"
    }
  }
  var json = JSON.parse(UrlFetchApp.fetch(url, options).getContentText()); 
  for(var i = 0; i < json.stats.length; i++) {
    if(json.stats[i].name === "goals") Logger.log(json.stats[i]);
  }
}

More Related Contents:

  1. Google Sheet use Importxml error could not fetch url [duplicate]
  2. How can I be getting multiple unwanted event blocks from the same onFormSubmit Trigger?
  3. Detect user inserting row or column in a google spreadsheet and reacting in a script
  4. Is there a custom function or script that returns gid of a specific sheet in Google Sheets?
  5. How to get onFormSubmit to trigger automatically?
  6. File movement between Google Shared Drive (team drive) folders using Apps Script
  7. Copying a spreadsheet copies all linked files as well
  8. Upload an image to s document
  9. How do I send the standard invitation email when calling addGuest on a CalendarEvent?
  10. “SpreadsheetApp.getUi() cannot be called from this context”
  11. Using built-in spreadsheet functions in a script
  12. Protect ranges with google apps script
  13. Get All Links in a Document
  14. Reference : TypeError: Cannot read property [property name here] from undefined
  15. Google script replaceAllShapesWithImage with image from drive doesn”t work any more
  16. ScriptApp.getService().getUrl() points to dev URL. How can I get it to point to exec production URL?
  17. Dynamically creating spreadsheet menu items
  18. Is there any limit on number of concurrent hits or simultaneous executions on Google App Script Web App
  19. Can I color certain words in Google Document using Google Apps Script?
  20. Creating a zip file inside Google Drive with Apps Script
  21. How do I know if spreadsheet cells are merged using google apps script
  22. How to ignore empty cell values for getRange().getValues()
  23. How do you tell whether text fits within its bounding shape?
  24. Google apps script error: “You do not have permission to call protect” [duplicate]
  25. Stackdriver Logging API returns response code 200, but response is empty
  26. TypeError: Cannot read property ‘getRange’ of null (line 6, file “Code”) in google sheet script [duplicate]
  27. Google spreadsheet “Cannot call method “getRange” of null”
  28. .setFormula() and received error missing ) argument list
  29. How to trigger a Google Apps Script once an email get in the inbox?
  30. Converting string to web-safe Base64 format

Leave a Comment