If you are using Selenium WebDriver as a web crawler and thinking that it's too slow, welcome inside! In this article, we will see how to make page parsing time around 50 times faster. As an example, I will parse comments from another article from this blog. I will first parse it using default WebDriver API (FindElement... methods) and then will compare it to CsQuery Here is WebDriver parsing code: var driver = new ChromeDriver(); driver.Navigate().GoToUrl("/2014/07/fixed-setup-was-unable-t...

If you want to run C# application (mono) with Selenium ChomeDriver on Ubuntu Server 16.04 in headless mode, you definitely should read this article. We will use Xvfb as X server, this will let us emulate "headless" mode because Xvfb performs all graphical operations in memory without showing any screen output. Install mono sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF echo "deb http://download.mono-project.com/repo/debian whe...