All Versions
5
Latest Version
Avg Release Cycle
-
Latest Release
-

Changelog History

  • v0.3.0 Changes

    • 👍 Allow scraping from multiple URLs in the same crawler.
    // This
fn new(url: String) -> Self {}

// Now becomes
fn new(urls: impl Into<CrawlerSource>) -> Self {}

// Which enables
Crawler::new(vec!["url1".into(), "url2".into()]);
Crawler::new("url".to_owned());

  • v0.2.1 Changes

    • The PreFetchCallback was not being called before fetching HEAD requests.

  • v0.2.0 Changes

    • Switch to using Arc<Fn()> callbacks instead of function pointers.

  • v0.1.1 Changes

    • ✂ Remove dependency on the failure crates
    • ➕ Add the content_type field to UrlEntry::File

  • v0.1.0 Changes

    • 🎉 Initial release