Lab 6 : Link Checking Telescope

This week, we put our link checking program to use with telescope, an aggregated blogging site for all of the current and past OSD600 students dev blogs. Last week I blogged about working on the site by contributing to an issue. Our professor tasked us this week to check links for the 10 latest posts that appears in telescope using our link checking program from earlier this semester.

So unlike the other students that haven't worked on telescope yet, which you can read about 2 posts back, myself (and a few others I expect) have already got telescope setup and have long since beaten that huge struggle. I'd look at the slack chat and I see everyone running into problems left and right, and man do I really feel for them. Took me a while to get it running initially, but fortunately I can go straight to implementing the link checking feature in my program.

So I thought writing the feature would be fairly simple, and for the most part it was. First, using the api with the url 'localhost:3000/posts' I fetch a json array with the post ids for the first 10 posts. I had to query it by adding 'posts?per_page=10' in the end of the url to filter it down since it defaults to 30 posts. Then using the ids, I use fetch again to retrieve the page data, which I then pass on to my link checking function, there it retrieves the links through regex filtering, checks each one of them and outputs their results.

The final code block

However, I did run into a problem, and it was regarding some asynchronous code problems, which I've always been so confused about. And it led me down a java script asynchronous tutorial rabbit hole. I was trying to understand how the event loop works, how promises are queued and the order synchronous tasked are carried out, and I don't fully get it 100%, but I think I got it enough. My htmlVerify function uses a Promise.All to process fetching the links concurrently and outputs it. Well I tried console logging before calling htmlVerify, but the problem was that the console log line is synchronous and will process first before any of the links are outputted since promises are queued to the event loop. 

So I fixed the problem by just console logging the post id as part of the output function that executes as part of the Promise.all chain. Then I got it working.

So then I generated a .diff file to post into gist to note the additions to my code and submitted my lab.

The gist file can be found here

[https://gist.github.com/joelazwar/0e14425eb3f6fc6e01e9ea9c8bdb34b3]

Thanks for reading