Errors scraping a web page and downloading files in the web service

Errors scraping a web page and downloading files in the web serviceFollow
1. Errors scraping a web page and downloading files in the web service

Bronze Contributor
James Watts
Posted Mar 02, 2023 06:44 AM
This works in the desktop app, but not on the service:

Web.BrowserContents call to https://www.data.boem.gov/Main/OGOR-A.aspx to scrape the web page
extract the list of file names from the a table in the web page
step through the list of file names, and retrieve each with Web.Contents, then process them
This does not work when I publish to the web – the Web.Contents is dynamic, so does not refresh as a Dataset.

Tried converting to a Dataflow, but all I get is cnxn errors, or at best if I wrap a Web.Page around the Web.BrowserContents, an empty page.

Any suggestions on how to first scrape a page and then how to download files in a Dataflow ?
(Or how to get it working in a Dataset ?)

——————————
James Watts
Business Intelligence Analyst
——————————