dimitris kalamaras

math, social network analysis, web dev, free software…

Tag: Web Crawler

SocNetV 3.0

SocNetV v3.0 is here!

The Social Network Visualizer project has released a brand-new version of our favorite SNA application. SocNetV v3.0 brings improved speed and responsiveness, bugfixes and new command line parameters.

Read More

SocNetV v1.6 released – a nice web crawler included!

The Social Network Visualizer project has just released its latest version 1.6. This new version brings back the web crawler feature, which has been disabled in the 1.x series so far, but in a much more improved form.

To start the web crawler, go to menu Network > Web Crawler or press Shift+C…

Read More

SocNetV 0.70 – the web crawler release :)

It’s been only one month since the release of 0.60, but Social Network Visualizer version 0.70 is here! This summertime release justifies its existence with two new features, a built-in web crawler and multiple node selection.

The web crawler, based on some older tutorial code I had, is pretty simple software (don’t expect Google power here!), which automatically creates mathematical graphs starting with a given URL, called the seed. As the algorithm crawls that seed webpage, it identifies all the links inside the page and adds them to a list of URLs (called frontier). Then, all the URLs from the frontier are recursively visited. In this process, a network of pages (nodes) and links (edges) is being developed allowing the researcher to study and visualize network properties, such as centralities, etc.

At the moment, the web crawler code is quite immature (i.e. doesn’t recognize framesets) but I hope it will evolve in the future. To test it, go to menu Network > Web Crawler (keyboard junkies, press Shift+C). A dialog will appear – enter the initial web address (seed), the maximum recursion level (how many URLs from the frontier will be visited) and the maximum running time and press OK. Here’s a screenshot of SocNetV crawling http://socnetv.org with betweenness centrality radial layout:

Read More

Powered by WordPress & Theme by Anders Norén