Protected resources

Alan AI allows you to crawl web resources protected with Basic authentication. To be able to retrieve data from such resources, add the auth parameter to the corpus() function and define the username and password to access the resource.

For example:

Dialog script

corpus({
    title: `Protected resources`,
    urls: [`https://testpages.eviltester.com/styled/auth/basic-auth-results.html`],
    auth: {username: `authorized`, password: `password001`},
    depth: 3,
    maxPages: 1,
    priority: 1,
});