Python Module

urllib
python3에서 urllib2는 urllib에 흡수되었다.

urllib는 다음 모듈을 모아 놓은 패키지이다.

  • urllib.request for opening and reading URLs
  • urllib.error containing the exceptions raised by urllib.request
  • urllib.parse for parsing URLs
  • urllib.robotparser for parsing robots.txt files

urllib는 url처리 모듈이다.
다음은 가장 간단한 사용방법이다.

# code from https://docs.python.org/3/howto/urllib2.html
import urllib.request
url = 'http://dna.daum.net'

response = urllib.request.urlopen(url) # file open과 유사
html = response.read()
response.close()

urlopen()은 url 또는 Request객체를 인자로 받는다. Request 객체를 사용하면, POST mehtod를 사용하거나, header를 추가할 수 있다. urlopen()에 data를 넘기면 POST method로 수행된다.

import urllib.request
url = 'http://dna.daum.net'
data = urllib.parse.urlencode({'spam':1, 'eggs':2})
data = data.encode('utf-8')
req1 = urllib.request.Request(url)
response = urllib.request.urlopen(url, data)
urllib.request
출처 : https://docs.python.org/3/library/urllib.request.html#module-urllib.request

urllib.request는 URL관련한 열기, 인증, 쿠키 등에 대한 클래스와 함수를 정의한다.

urllib.parse

# Class
urllib.request.Request(url, data=None, headers={}, origin_req_host=None, unverifiable=False, method=None)

# Function
urllib.request.urlopen(url, data=None, [timeout, ]*, cafile=None, capath=None, cadefault=False)

기본 사용형태는 다음과 같다

import urllib.request

url = "http://www.naver.com"

req = urllib.request.Request(url)
response = urllib.request.urlopen(req)

result = response.read().decode('utf-8')

BeautifulSoup4

Reference

기본 사용방법은 html문서를 BeautfilSoup에 넣어주면 된다.

import bs4

soup = bs4.BeautifulSoup(html_doc)

print(soup.prettify())  # 보기 좋게 출력

soup.title        # title tag에 접근
soup.body        # body tag에 접근

for talk in soup.find_all('div', {'class':'talk'}):  # div tag의 속성으로 검색
    print(talk.span.text)
# <div class=\'talk\'><span>Mom: How was school today, Sam?</span></div>

find_all()의 결과는 ResultSet인데, 각 결과를 담고 있는 list이다. 따라서 list의 원소에 대해 다시 find_all()을 사용가능하다. 일출 시간을 가져오는 예제이다. (https://github.com/hyde1004/sunrise_info)

url = 'http://astro.kasi.re.kr/Life/Knowledge/sunmoon_map/sunmoon_popup.php?year=2014&month=9&location=%C3%B5%BE%C8'

soup = bs4.BeautifulSoup(html)

day_info = soup.tbody.find_all('tr')
for info in day_info:
    sunrise_info = info.find_all('td')
    print(sunrise_info[2].text)

unittest

Skipping tests

@unittest.skip( )을 사용하면, 해당 test를 skip할 수 있다. 이외에 여러 조건을 지정할 수 있다.

class MyTestCase(unittest.TestCase):
    @unittest.skip("demonstrating skipping")
    def test_nothing(self):
        self.fail("shouldn't happen")

'Python' 카테고리의 다른 글

matplotlib 이용하기  (0) 2014.09.15
Python3와 encoding  (0) 2014.08.30
Python Note  (0) 2014.07.24
Code Academy - Python  (0) 2014.07.11

Python Note

정수 나눗셈
3 / 2 # 1.5
3 // 2 # 1
List Comprehension

아래에서 결국엔 len(n)으로 List가 채워진다.

len_cycles = [len(n) for n in cycles]
Directory operators

https://docs.python.org/3.4/library/pathlib.html#basic-use

zip function

여러 리스트를 김밥말듯 말아서, 자른다.

a = [1, 2, 3, 4, 5]
b = ['a', 'e', 'i', 'o', 'u']
for x, y in zip(a, b):
    print(x, y)

1 a
2 e
3 i
4 o
5 u
import 사용법
# import 파일명 형태
import module1
module1.hello()

# from 파일명 import 함수명 형태 (파일명 생략가능)
from module1 import hello()

if __name__ == '__main__' 구문은 파일 단독으로 실행하는 경우만 동작하고, 다른 파일에서 재사용할때는 동작하지 않는 부분이다.

list slice
출처 : http://codingbat.com/prob/p148661

다음의 결과는 서로 다르다.

def rotate_left3(nums):
    return nums[1:] + nums[0:1] # list + list
    return nums[1:] + nums[0] # 불가 : list + int
진수 관련

10진수를 제외한 진수은 0을 붙이고 시작한다.

a = 10     # 10진수 10
b = 0b10   # 2진수 10, 10진수 2
c = 0o10   # 8진수 10, 10진수 8
d = 0x10   # 16진수 10, 10진수 16

10진수를 다른 진수로 변환하기

bin(10)    # '0b1010'
oct(10)    # '0o12'
hex(10)    # '0xa'

문자를 해당 진수로 해석하기

int("10")     # 10
int("10", 2)  # 2
int("10", 8)  # 8
int("10", 16) # 16

'Python' 카테고리의 다른 글

matplotlib 이용하기  (0) 2014.09.15
Python3와 encoding  (0) 2014.08.30
Python Module  (0) 2014.07.31
Code Academy - Python  (0) 2014.07.11
Reference : http://www.codecademy.com/tracks/python

Python Syntax

Multi-Line Comments
"""Sipping from your cup 'til it runneth over,
Holy Grail.
"""

Strings & Console Output

Escaping characters
'There\'s a snake in my boot!'
String methods
  • len()
  • lower()
  • upper()
  • str()
"Ryan".lower()
"Ryan".upper()
str(3.14)
len("roar")
String Formatting with %
print "The %s who %s %s!" % ("Knights", "say", "Ni")
# This will print "The Knights who say Ni!"

Date and Time

Getting the Current Date and Time
from datetime import datetime

now = datetime.now()
print now
Extracting Information
from datetime import datetime
now = datetime.now()

current_year = now.year
current_month = now.month
current_day = now.day
Pretty Time
from datetime import datetime
now = datetime.now()

print '%s:%s:%s' % (now.hour, now.minute, now.second)

Conditionals & Control Flow

And
  • 1 < 2 and 2 < 3 is True
  • 1 < 2 and 2 > 3 is False
This and That (or This, But Not That!)

Boolean operators aren’t just evaluated from left to right. Just like with arithmetic operators, there’s an order of operations for boolean operators:

  • not is evaluated first
  • and is evaluated next
  • or is evaluated last
The Big If
if this_might_be_true():
    print "This really is true."
elif that_might_be_true():
    print "That is true."
else:
    print "None of the above."

PygLatin

Input!
name = raw_input("What's your name?")
print name
Check Yourself… Some More
x = "J123"
x.isalpha()  # False

Functions

Generic Imports
# Ask Python to print sqrt(25) on line 3.
import math

print math.sqrt(25)
Function Imports
from module import function

print sqrt(25)
Universal Imports
from module import *
Here Be Dragons
import math            # Imports the math module
everything = dir(math) # Sets everything to a list of things from math
print everything       # Prints 'em all!
['__doc__', '__name__', '__package__', 'acos', 'acosh', 'asin', 'asinh', 'atan', 'atan2', 'atanh', 'ceil', 'copysign', 'cos', 'cosh', 'degrees', 'e', 'erf', 'erfc', 'exp', 'expm1', 'fabs', 'factorial', 'floor', 'fmod', 'frexp', 'fsum', 'gamma', 'hypot', 'isinf', 'isnan', 'ldexp', 'lgamma', 'log', 'log10', 'log1p', 'modf', 'pi', 'pow', 'radians', 'sin', 'sinh', 'sqrt', 'tan', 'tanh', 'trunc']
None
On Beyond Strings
def biggest_number(*args):
    print max(args)
    return max(args)

biggest_number(-10, -5, 5, 10)
Built-in Functions
maximum = max(1, 1.1, -3.2)
print maximum

minimum = min(-100, 0, 44, 100.3)
print minimum

absolute = abs(-42)
print absolute

print type(4)     # <type 'int'>
print type(4.2)   # <type 'float'>
print type('hello') # <type 'str'>

Python Lists and Dictionaries

Introduction to Lists

Lists are a datatype you can use to store a collection of different pieces of information as a sequence under a single variable name.

Late Arrivals & List Length
letters = ['a', 'b', 'c']
letters.append('d')
print len(letters)
print letters
List Slicing
suitcase = ["sunglasses", "hat", "passport", "laptop", "suit", "shoes"]

first  = suitcase[0:2]  # The first and second items (index zero and one)
middle = suitcase[2:4]  # Third and fourth items (index two and three)
last   = suitcase[4:6]  # The last two items (index four and five)
Slicing Lists and Strings
animals = "catdogfrog"
cat  = animals[:3]   # The first three characters of animals
dog  = animals[3:6]              # The fourth through sixth characters
frog = animals[6:]              # From the seventh character to the end
Maintaining Order
animals = ["ant", "bat", "cat"]
print animals.index("bat")

animals.insert(1, "dog")
print animals
For One and All
my_list = [1,9,3,8,5,7]

for number in my_list:
    # Your code here
    print 2 * number
More with ‘for’
animals = ["cat", "ant", "bat"]
animals.sort()

for animal in animals:
    print animal
This Next Part is Key
# Assigning a dictionary with three key-value pairs to residents:
residents = {'Puffin' : 104, 'Sloth' : 105, 'Burmese Python' : 106}

print residents['Puffin'] # Prints Puffin's room number

# Your code here!
print residents['Sloth']
print residents['Burmese Python']
New Entries
dict_name[new_key] = new_value
Changing Your Mind
del dict_name[key_name]

dict_name[key] = new_value
Remove a Few Things
beatles = ["john","paul","george","ringo","stuart"]
beatles.remove("stuart")
print beatles
>> ["john","paul","george","ringo"]
It’s Dangerous to Go Alone! Take This
my_dict = {
    "fish": ["c", "a", "r", "p"],
    "cash": -4483,
    "luck": "good"
}
print my_dict["fish"][0]

A Day at the Supermarket

This is KEY!
# A simple dictionary
d = {"foo" : "bar"}

for key in d: 
    print d[key]  # prints "bar"

Student Becomes the Teacher

Lesson Number One
animal_sounds = {
    "cat": ["meow", "purr"],
    "dog": ["woof", "bark"],
    "fox": [],
}
print animal_sounds["cat"]

Lists and Functions

Appending to a list
n = [1, 3, 5]
# Append the number 4 here
n.append(4)
Removing elements from lists

This exercise will expand on ways to remove items from a list. You actually have a few options. For a list called n:

  1. n.pop(index) will remove the item at index from the list and return it to you:
    n = [1, 3, 5]
    n.pop(1)
    # Returns 3 (the item at index 1)
    print n
    # prints [1, 5]
    
  2. n.remove(item) will remove the actual item if it finds it:
    n.remove(1)
    # Removes 1 from the list,
    # NOT the item at index 1
    print n
    # prints [3, 5]
    
  3. del(n[1]) is like .pop in that it will remove the item at the given index, but it won’t return it:
    del(n[1])
    # Doesn't return anything
    print n
    # prints [1, 5]
    
Passing a range into a function
range(6) # => [0,1,2,3,4,5]
range(1,6) # => [1,2,3,4,5]
range(1,6,3) # => [1,4]
  • range(stop)
  • range(start, stop)
  • range(start, stop, step)
Iterating over a list in a function

Now that we’ve learned about range, we have two ways of iterating through a list.

  • Method 1 : for item in list

    for item in list:
     print item
    
  • Method 2 : iterate through indexes

    for i in range(len(list)):
     print list[i]
    

Method 1 is useful to loop through the list, but it’s not possible to modify the list this way. Method 2 uses indexes to loop through the list, making it possible to also modify the list if needed. Since we aren’t modifying the list, feel free to use either one on this lesson!

Using two lists as two arguments in a function
a = [1, 2, 3]
b = [4, 5, 6]
print a + b
# prints [1, 2, 3, 4, 5, 6]

Battleship!

Make a List
print ["O"] * 5
# prints ['O', 'O', 'O', 'O', 'O']
Printing Pretty
letters = ['a', 'b', 'c', 'd']
print " ".join(letters)
print "---".join(letters)
Hide…
from random import randint
coin = randint(0, 1) # 0, 1
dice = randint(1, 6) # 1, 2, 3, 4, 5, 6

Loops

While / else
while condition:
    pass
else:
    pass
For your strings
word = "eggs!"
for c in word:
    print c
Looping over a dictionary
d = {'x': 9, 'y': 10, 'z': 20}
for key in d:
    if d[key] == 10
        print "This dictionary has the value 10!"
Counting as you go
choices = ['pizza', 'pasta', 'salad', 'nachos']

print 'Your choices are:'
for index, item in enumerate(choices):
    print index + 1, item
Multiple lists

It’s also common to need to iterate over two lists at once. This is where the built-in zip function comes in handy.

zip will create pairs of elements when passed two lists, and will stop at the end of the shorter list.

zip can handle three or more lists as well!

For / else
names = ['Hyde', 'Jack', 'Dorothy', 'Jane']

for name in names:
    print name
else:
    print "No more name"

Advanced Topics in Python

keys() and values()
my_dict = {
    "Name" : "Junggu Lee",
    "Company" : "LGE",
    "Age" : 41
}

print my_dict.keys()
print my_dict.values()
print my_dict.items()
The ‘in’ Operator
for number in range(5):
    print number,

d = { "name": "Eric", "age": 26 }
for key in d:
    print key, d[key],

for letter in "Eric":
    print letter,  # note the comma!
List Comprehension Syntax
new_list = [x for x in range(1,6)]
# => [1, 2, 3, 4, 5]

doubles = [x*2 for x in range(1,6)]
# => [2, 4, 6, 8, 10]

doubles_by_3 = [x*2 for x in range(1,6) if (x*2)%3 == 0]
# => [6]
List Slicing Syntax
[start:end:stride]
Reversing a List
letters = ['A', 'B', 'C', 'D', 'E']
print letters[::-1]
Anonymous Functions

See the lambda bit? Typing

lambda x: x % 3 == 0

Is the same as

def by_three(x):
    return x % 3 == 0
Lambda Syntax
my_list = range(16)
print filter(lambda x: x % 3 == 0, my_list)
Try it!
cubes = [x**3 for x in range(1, 11)]
filter(lambda x: x % 3 == 0, cubes)

Binary Representaion

Lesson 10:The Base 2 Number System
print 0b1,    #1
print 0b10,   #2
The bin() Function
  • bin() function
  • oct() function
  • hex() function
print bin(1)
print bin(2)
int()’s Second Parameter
int("42")
# ==> 42

int("110", 2)
# ==> 6

Introduction to Classes

Class Syntax
class NewClass(object):
    # Class magic here
    pass
Let’s Not Get Too Selfish
class Animal(object):
    def __init__(self, name):
        self.name = name
Inheritance Syntax
class DerivedClass(BaseClass):
    # code goes here
Override!
class Employee(object):
    def __init__(self, name):
        self.name = name
    def greet(self, other):
        print "Hello, %s" % other.name

class CEO(Employee):
    def greet(self, other):
        print "Get back to work, %s!" % other.name

ceo = CEO("Emily")
emp = Employee("Steve")
emp.greet(ceo)
# Hello, Emily
ceo.greet(emp)
# Get back to work, Steve!

``` This Looks Like a Job For …

class Derived(Base):
   def m(self):
       return super(Derived, self).m()

File Input/Output

See It to Believe It
my_list = [i**2 for i in range(1,11)]
# Generates a list of squares of the numbers 1 - 10

f = open("output.txt", "w")

for item in my_list:
    f.write(str(item) + "\n")

f.close()
Reading
my_file = open("output.txt", "r")
print my_file.read()
my_file.close()
Reading Between the Lines
my_file = open("text.txt", "r")

print my_file.readline()
print my_file.readline()
print my_file.readline()

my_file.close()
The ‘with’ and ‘as’ Keywords

Programming is all about getting the computer to do the work. Is there a way to get Python to automatically close our files for us?

Of course there is. This is Python.

You may not know this, but file objects contain a special pair of built-in methods: __enter__() and __exit__(). The details aren’t important, but what is important is that when a file object’s __exit__() method is invoked, it automatically closes the file. How do we invoke this method? With with and as.

The syntax looks like this:

with open("file", "mode") as variable:
    # Read or write to the file
with open("text.txt", "w") as textfile:
    textfile.write("Success!")
Case Closed?
f = open("bg.txt")
f.closed
# False
f.close()
f.closed
# True

'Python' 카테고리의 다른 글

matplotlib 이용하기  (0) 2014.09.15
Python3와 encoding  (0) 2014.08.30
Python Module  (0) 2014.07.31
Python Note  (0) 2014.07.24

+ Recent posts