Python Virtual Environment Wrapper

$ pip install virtualenvwrapper
$ which virtualenvwrapper.sh
/usr/local/bin/virtualenvwrapper.sh

Adding path

$ export WORKON_HOME=$HOME/.virtualenvs   # Optional
$ export PROJECT_HOME=$HOME/projects      # Optional
$ source /usr/local/bin/virtualenvwrapper.sh

Reloading Shell

$ source ~/.bashrc
$ echo $WORKON_HOME
/Users/mervinpraison/.virtualenvs

Functions

$ mkvirtualenv my-new-project
(my-new-project) $

To stop using the environment

(my-new-project) $ deactivate
$
$ workon
my-new-project

To activate the environment

$ workon my-new-project

-p parameter to choose the Python Version

$ virtualenv -p $(which python3) blog_virtualenv

Python Redirect Path

import requests 
s = requests.Session()
response = s.get("https://google.com/")
cookies = dict(response.cookies)
if response.history:
    print ("Request was redirected")
    for resp in response.history:
        print (resp.status_code, resp.url)
    print ("Final destination:")
    print (response.status_code, response.url)
else:
    print ("Request was not redirected")

Selenium Firefox Webdriver Python Setup

Python Code

from bs4 import BeautifulSoup
import requests
import pandas as pd
import numpy as np
import csv
import re
from selenium import webdriver


#driver = webdriver.Firefox(capabilities={"marionette":False})
caps = webdriver.DesiredCapabilities.FIREFOX
caps["marionette"] = False
driver = webdriver.Firefox(capabilities=caps)


driver.get("https://www.google.com")

print (driver.title)

Python Code with Headless Firefox

from bs4 import BeautifulSoup
import requests
import pandas as pd
import numpy as np
import csv
import re
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
options = Options()
options.add_argument("--headless")


#driver = webdriver.Firefox(capabilities={"marionette":False})
caps = webdriver.DesiredCapabilities.FIREFOX
caps["marionette"] = False
driver = webdriver.Firefox(capabilities=caps, firefox_options=options)


driver.get("https://www.google.com")

print (driver.title)

WebDriverException: Message: Can’t load the profile

Try,

caps["marionette"] = True

Get all Divs using Selenium Driver Python

X Path

divs = driver.find_elements_by_xpath('//li/div')

CSS Selector

divs = driver.find_elements_by_css_selector('li > div')