正确安装python talib库

1.环境 win7,64位, anaconder,pyhon2.7,3.5.。网上找来很多方法不管用。

2. 用anaconda装也不行。最后是从这个帖子,发现好像官网的是32位,我们要下64位的:
    http://stackoverflow.com/questions/28103010/error-installing-ta-lib-for-anaconda
    找到这个link:
    http://www.lfd.uci.edu/~gohlke/pythonlibs/
    从那边下载的这个:

    TA_Lib-0.4.17-cp35-cp35m-win_amd64.whl

    TA_Lib-0.4.17-cp27-cp27m-win_amd64.whl

3. 然后在anaconder python2.7 的命令行下: 

    pip install TTA_Lib-0.4.17-cp27-cp27m-win_amd64.whl,显示安装成功,但实际在python2.7的环境下,

    import talib会明显出错。 报:“ImportError: cannot import name __TA_FUNCTION_NAMES__”

    不知原因。

4. 然后在anaconder python3.5 的命令行下: 

    pip install TA_Lib-0.4.9-cp35-none-win_amd64.whl

    实测通过。

如何将 WordPress 博客发布到静态 GitLab Pages 上

很久以前,我为一个家庭成员建立了一个 WordPress 博客。如今有很多选择,但是当时如果你需要一个带有所见即所得的编辑器的基于 Web 的 CMS,那么就没什么像样的的选择了。而一切运行良好的不幸的副作用是随着时间的推移该博客产生了很多内容。这意味着我要经常更新 WordPress 以防止不断出现的漏洞。

因此,当我决定劝说家人切换到 Hugo[1] 会相对容易,然后可以在 GitLab[2] 上托管博客。但是尝试提取所有内容并将其转换为 Markdown[3] 变成了一个巨大的麻烦。有自动脚本完成了 95% 的工作,但并不完美。手动更新所有帖子不是我想做的事情,所以最终,我放弃了试图移动博客。

最近,我又开始考虑这个问题,并意识到有一个我没有考虑过的解决方案:我可以继续维护 WordPress 服务器,但将其设置为发布静态镜像,并使用 GitLab Pages[4](或 GitHub Pages[5] ,如果你喜欢的话)提供服务。这能让我自动化 Let’s Encrypt[6] 证书续订并消除与托管 WordPress 站点相关的安全问题。然而,这意味着评论将无法使用,但在这种情况下感觉就像是一个小损失,因为博客没有收到很多评论。

这是我提出的解决方案,到目前为止似乎运作良好:

托管 WordPress 站点中的 URL 没有链接到或来自其他任何地方,以减少它被利用的几率。在此例中,我们将使用 http://private.localconspiracy.com(即使此站点实际上是使用 Pelican 构建的)。
将公共 URL https://www.localconspiracy.com 托管到 GitLab Pages 上[7]。
添加 cron 任务[8],确定两个 URL 之间的最后构建日期何时不同。如果构建日期不同,则镜像 WordPress 版本。
使用 wget 镜像后,将所有链接从“私有”更新成“公共”。
运行 git push 来发布新内容。
这是我使用的两个脚本:

check-diff.sh (cron 每 15 分钟调用一次):

#!/bin/bash

ORIGINDATE=”$(curl -v –silent http://private.localconspiracy.com/feed/ 2>&1|grep lastBuildDate)”
PUBDATE=”$(curl -v –silent https://www.localconspiracy.com/feed/ 2>&1|grep lastBuildDate)”

if [ “$ORIGINDATE” != “$PUBDATE” ]
then
/home/doc/repos/localconspiracy/mirror.sh
fi
mirror.sh:

#!/bin/sh

cd /home/doc/repos/localconspiracy

wget \
–mirror \
–convert-links \
–adjust-extension \
–page-requisites \
–retry-connrefused \
–exclude-directories=comments \
–execute robots=off \
http://private.localconspiracy.com

git rm -rf public/*
mv private.localconspiracy.com/* public/.
rmdir private.localconspiracy.com
find ./public/ -type f -exec sed -i -e ‘s|http://private.localconspiracy|https://www.localconspiracy|g’ {} \;
find ./public/ -type f -exec sed -i -e ‘s|http://www.localconspiracy|https://www.localconspiracy|g’ {} \;
git add public/*
git commit -m “new snapshot”
git push origin master
就是这些了!现在,当博客发生变化时,在 15 分钟内将网站镜像到静态版本并推送到仓库,这将在 GitLab Pages 中反映出来。

如果你想在本地运行 WordPress[9],这个概念可以进一步扩展。在这种情况下,你不需要服务器来托管你的 WordPress 博客。你可以在本机运行它。在这种情况下,你的博客不可能被攻击利用。只要你可以在本地运行 wget,就可以使用上面的方法在 GitLab Pages 上托管 WordPress 站点。

获取股票交易数据的Tushare的使用方法

Tushare是一个免费、开源的python财经数据接口包。主要实现对股票等金融数据从数据采集、清洗加工 到 数据存储的过程,能够为金融分析人员提供快速、整洁、和多样的便于分析的数据,为他们在数据获取方面极大地减轻工作量,使他们更加专注于策略和模型的研究与实现上。考虑到Python pandas包在金融量化分析中体现出的优势,Tushare返回的绝大部分的数据格式都是pandas DataFrame类型,非常便于用pandas/NumPy/Matplotlib进行数据分析和可视化。当然,如果您习惯了用Excel或者关系型数据库做分析,您也可以通过Tushare的数据存储功能,将数据全部保存到本地后进行分析。应一些用户的请求,从0.2.5版本开始,Tushare同时兼容Python 2.x和Python 3.x,对部分代码进行了重构,并优化了一些算法,确保数据获取的高效和稳定。

不管你是量化投资分析师,还是正在学习Python进行数据分析的学习者,这种方法获取的数据都可以适用。

一、获取前的准备

1、安装Python,这个不再赘述,Python官网下载安装适合自己计算机的版本。

2、安装pandas,我是直接安装的Anaconda,包含有pandas,很实用,建议安装Anaconda,下载地址:https://www.anaconda.com/download/ ,选择适合的版本安装。

3、Tushare的下载安装,下载地址:https://pypi.python.org/pypi/tushare ,下载完之后解压即可,无需安装。

二、获取数据

1、获取历史行情数据 get_hist_data()

获取个股历史交易数据(包括均线数据),可以通过参数设置获取日k线、周k线、月k线,以及5分钟、15分钟、30分钟和60分钟k线数据。本接口只能获取近3年的日线数据,适合搭配均线数据进行选股和分析。

参数说明:

code:股票代码,即6位数字代码,或者指数代码(sh=上证指数 sz=深圳成指 hs300=沪深300指数 sz50=上证50 zxb=中小板 cyb=创业板)
start:开始日期,格式YYYY-MM-DD
end:结束日期,格式YYYY-MM-DD
ktype:数据类型,D=日k线 W=周 M=月 5=5分钟 15=15分钟 30=30分钟 60=60分钟,默认为D
retry_count:当网络异常后重试次数,默认为3
pause:重试时停顿秒数,默认为0
返回值说明:

date:日期
open:开盘价
high:最高价
close:收盘价
low:最低价
volume:成交量
price_change:价格变动
p_change:涨跌幅
ma5:5日均价
ma10:10日均价
ma20:20日均价
v_ma5:5日均量
v_ma10:10日均量
v_ma20:20日均量
turnover:换手率[注:指数无此项]
使用方法:

import tushare as ts
data=ts.get_hist_data(‘300274’)
print(data)
需要注意的是路径要在tushare文件下,否则会报错。

执行结果:

View Code
获取某一时间段的历史数据

import tushare as ts
data=ts.get_hist_data(‘300274′,start=’2017-01-01′,end=’2018-03-27’)
print(data)
执行结果:

View Code
另外一个获取历史数据的函数get_h_data()。

import tushare as ts
data=ts.get_h_data(‘300274’)
print(data)
执行结果:

View Code
在不指定开始时间和结束时间时,该函数默认返回最近一年的日线数据,返回的数据与get_hist_data不同的是,该函数只返回开盘价(open)、最高价(high)、收盘价(close)、最低价(low)、成交量(volume)、成交金额(amount)六列 ,同样加上时间段也可以获取相应数据。

第三个获取历史数据的函数get_k_data()。

import tushare as ts
data=ts.get_k_data(‘300274’)
print(data)
执行结果:

View Code
与前两个函数相比,这个函数获取数据的速度很明显要快很多,而且可以返回每一只股票从上市开始到当前交易日的所有日线数据,这个有点是前两个函数都不具备的,更重要的是,如果批量3000多只股票的数据,前两个都不如get_k_data()稳定。

其他:

复制代码
ts.get_hist_data(‘600848′, ktype=’W’) #获取周k线数据
ts.get_hist_data(‘600848′, ktype=’M’) #获取月k线数据
ts.get_hist_data(‘600848′, ktype=’5’) #获取5分钟k线数据
ts.get_hist_data(‘600848′, ktype=’15’) #获取15分钟k线数据
ts.get_hist_data(‘600848′, ktype=’30’) #获取30分钟k线数据
ts.get_hist_data(‘600848′, ktype=’60’) #获取60分钟k线数据
ts.get_hist_data(‘sh’)#获取上证指数k线数据,其它参数与个股一致,下同
ts.get_hist_data(‘sz’)#获取深圳成指k线数据
ts.get_hist_data(‘hs300’)#获取沪深300指数k线数据
ts.get_hist_data(‘sz50’)#获取上证50指数k线数据
ts.get_hist_data(‘zxb’)#获取中小板指数k线数据
ts.get_hist_data(‘cyb’)#获取创业板指数k线数据
复制代码
2、获取实时行情数据get_today_all()

返回值说明:

code:代码
name:名称
changepercent:涨跌幅
trade:现价
open:开盘价
high:最高价
low:最低价
settlement:昨日收盘价
volume:成交量
turnoverratio:换手率
amount:成交量
per:市盈率
pb:市净率
mktcap:总市值
nmc:流通市值
import tushare as ts
ts.get_today_all()
执行结果:

View Code
一次性获取当前交易所有股票的行情数据(如果是节假日,即为上一交易日)

3、获取历史分笔数据之:get_tick_data()

获取个股以往交易历史的分笔数据明细,通过分析分笔数据,可以大致判断资金的进出情况。在使用过程中,对于获取股票某一阶段的历史分笔数据,需要通过加入交易日参数并append到一个DataFrame或者直接append到本地同一个文件里。历史分笔接口只能获取当前交易日之前的数据,当日分笔历史数据请调用get_today_ticks()接口或者在当日18点后通过本接口获取。

参数说明:

code:股票代码,即6位数字代码
date:日期,格式YYYY-MM-DD
retry_count : int, 默认3,如遇网络等问题重复执行的次数
pause : int, 默认 0,重复请求数据过程中暂停的秒数,防止请求间隔时间太短出现的问题
使用方法:

import tushare as ts
df=ts.get_tick_data(‘300274′,date=’2018-03-27’)
df.head(10) #距离执行时间最近的十笔交易
执行结果:

复制代码
time price change volume amount type
0 15:00:03 18.46 — 2373 4380761 卖盘
1 14:57:03 18.46 -0.01 188 347048 卖盘
2 14:57:00 18.47 -0.01 42 77574 卖盘
3 14:56:57 18.48 — 644 1190112 买盘
4 14:56:54 18.48 0.01 80 147840 买盘
5 14:56:51 18.47 0.01 9 16623 中性盘
6 14:56:48 18.46 -0.02 38 70148 卖盘
7 14:56:45 18.48 0.01 25 46200 买盘
8 14:56:42 18.47 — 87 160689 买盘
9 14:56:36 18.47 — 82 151454 买盘
复制代码
获取当日历史分笔数据:get_today_ticks()

获取当前交易日(交易进行中使用)已经产生的分笔明细数据。

参数说明:

code:股票代码,即6位数字代码
retry_count : int, 默认3,如遇网络等问题重复执行的次数
pause : int, 默认 0,重复请求数据过程中暂停的秒数,防止请求间隔时间太短出现的问题
返回值说明:

time:时间
price:当前价格
pchange:涨跌幅
change:价格变动
volume:成交手
amount:成交金额(元)
type:买卖类型【买盘、卖盘、中性盘】
使用方法:

import tushare as ts
df = ts.get_today_ticks(‘300274’) #当日的历史分笔数据
df.head(10) #距离执行时间最近的10笔
执行结果:

复制代码
[Getting data:]###########################################Out[6]:
time price pchange change volume amount type
0 13:32:15 18.68 +1.19 -0.01 1 1868 买盘
1 13:32:12 18.69 +1.25 0.00 8 14952 买盘
2 13:32:06 18.69 +1.25 0.01 1 1869 买盘
3 13:32:03 18.68 +1.19 0.00 19 35492 卖盘
4 13:31:51 18.68 +1.19 0.00 8 14944 买盘
5 13:31:48 18.68 +1.19 0.01 14 26152 买盘
6 13:31:45 18.67 +1.14 -0.01 30 56010 卖盘
7 13:31:42 18.68 +1.19 0.01 8 14944 买盘
8 13:31:36 18.67 +1.14 0.00 14 26138 卖盘
9 13:31:33 18.67 +1.14 -0.01 10 18670 卖盘
复制代码
4、获取实时分笔数据之:get_realtime_quotes()

获取实时分笔数据,可以实时取得股票当前报价和成交信息,其中一种场景是,写一个python定时程序来调用本接口(可两三秒执行一次,性能与行情软件基本一致),然后通过DataFrame的矩阵计算实现交易监控,可实时监测交易量和价格的变化。

参数说明:

symbols:6位数字股票代码,或者指数代码(sh=上证指数 sz=深圳成指 hs300=沪深300指数 sz50=上证50 zxb=中小板 cyb=创业板) 可输入的类型:str、list、set或者pandas的Series对象
返回值说明:

复制代码
0:name,股票名字
1:open,今日开盘价
2:pre_close,昨日收盘价
3:price,当前价格
4:high,今日最高价
5:low,今日最低价
6:bid,竞买价,即“买一”报价
7:ask,竞卖价,即“卖一”报价
8:volume,成交量 maybe you need do volume/100
9:amount,成交金额(元 CNY)
10:b1_v,委买一(笔数 bid volume)
11:b1_p,委买一(价格 bid price)
12:b2_v,“买二”
13:b2_p,“买二”
14:b3_v,“买三”
15:b3_p,“买三”
16:b4_v,“买四”
17:b4_p,“买四”
18:b5_v,“买五”
19:b5_p,“买五”
20:a1_v,委卖一(笔数 ask volume)
21:a1_p,委卖一(价格 ask price)

30:date,日期;
31:time,时间;
复制代码
使用方法:

import tushare as ts
df = ts.get_realtime_quotes(‘300274’) #单个股票实时行情
df[[‘code’,’name’,’price’,’bid’,’ask’,’volume’,’amount’,’time’]] #需要显示的属性
执行结果:

code name price bid ask volume amount time
0 300274 阳光电源 18.780 18.750 18.770 17003279 318581216.230 13:16:06

多只股票的实时分笔数据(最好不要超过30):

import tushare as ts
df = ts.get_realtime_quotes([‘600460′,’000762′,’000725’])
df[[‘code’,’name’,’price’,’bid’,’ask’,’volume’,’amount’,’time’]]
获取实时指数:

ts.get_realtime_quotes(‘sh’) #上证指数

ts.get_realtime_quotes([‘sh’,’sz’,’hs300′,’sz50′,’zxb’,’cyb’]) #上证指数 深圳成指 沪深300指数 上证50 中小板 创业板

ts.get_realtime_quotes([‘sh’,’600848′]) #或者混搭
大盘指数行情列表:

获取大盘指数实时行情列表,以表格的形式展示大盘指数实时行情。

返回值说明:

code:指数代码
name:指数名称
change:涨跌幅
open:开盘点位
preclose:昨日收盘点位
close:收盘点位
high:最高点位
low:最低点位
volume:成交量(手)
amount:成交金额(亿元)
使用方法:

import tushare as ts
df = ts.get_index()
print(df)
执行结果:

View Code
5、获取大单交易数据

获取大单交易数据,默认为大于等于400手,数据来源于新浪财经。

参数说明:

code:股票代码,即6位数字代码
date:日期,格式YYYY-MM-DD
vol:手数,默认为400手,输入数值型参数
retry_count : int, 默认3,如遇网络等问题重复执行的次数
pause : int, 默认 0,重复请求数据过程中暂停的秒数,防止请求间隔时间太短出现的问题
返回值说明:

code:代码
name:名称
time:时间
price:当前价格
volume:成交手
preprice :上一笔价格
type:买卖类型【买盘、卖盘、中性盘】
使用方法:

import tushare as ts
df = ts.get_sina_dd(‘300274′, date=’2018-03-28’) #默认400手
#df = ts.get_sina_dd(‘300274′, date=’2018-03-28’, vol=500) #指定大于等于500手的数据
print(df)
执行结果:

View Code
6、数据的存储

数据存储模块主要是引导用户将数据保存在本地磁盘或数据库服务器上,便于后期的量化分析和回测使用,在以文件格式保存在电脑磁盘的方式上,调用的是pandas本身自带的方法,此处会罗列常用的参数和说明。

保存为CSV格式文件

pandas的DataFrame和Series对象提供了直接保存csv文件格式的方法,通过参数设定,轻松将数据内容保存在本地磁盘。

常用参数说明:

path_or_buf: csv文件存放路径或者StringIO对象
sep : 文件内容分隔符,默认为,逗号
na_rep: 在遇到NaN值时保存为某字符,默认为’‘空字符
float_format: float类型的格式
columns: 需要保存的列,默认为None
header: 是否保存columns名,默认为True
index: 是否保存index,默认为True
mode : 创建新文件还是追加到现有文件,默认为新建
encoding: 文件编码格式
date_format: 日期格式
注:在设定path时,如果目录不存在,程序会提示IOError,请先确保目录已经存在于磁盘中。

使用方法:

复制代码
import tushare as ts
df = ts.get_hist_data(‘300274’)
#直接保存
df.to_csv(‘C:/Users/07010018/Desktop/tushare-1.1.6/exportdata/300274.csv’)
#选择保存
#df.to_csv(‘C:/Users/07010018/Desktop/tushare-1.1.6/exportdata/300274.csv’,columns=[‘open’,’high’,’low’,’close’])
复制代码
追加数据的方式:

某些时候,可能需要将一些同类数据保存在一个大文件中,这时候就需要将数据追加在同一个文件里,简单举例如下:

复制代码
import tushare as ts
import os

filename = ‘c:/day/bigfile.csv’
for code in [‘000875’, ‘600848’, ‘000981’]:
df = ts.get_hist_data(code)
if os.path.exists(filename):
df.to_csv(filename, mode=’a’, header=None)
else:
df.to_csv(filename)
复制代码
【注:如果是不考虑header,直接df.to_csv(filename, mode=’a’)即可,否则,每次循环都会把columns名称也append进去】

保存为Excel格式文件

pandas将数据保存为MicroSoft Excel文件格式。

常用参数说明:

excel_writer: 文件路径或者ExcelWriter对象
sheet_name:sheet名称,默认为Sheet1
sep : 文件内容分隔符,默认为,逗号
na_rep: 在遇到NaN值时保存为某字符,默认为’‘空字符
float_format: float类型的格式
columns: 需要保存的列,默认为None
header: 是否保存columns名,默认为True
index: 是否保存index,默认为True
encoding: 文件编码格式
startrow: 在数据的头部留出startrow行空行
startcol :在数据的左边留出startcol列空列
使用方法:

复制代码
import tushare as ts
df = ts.get_hist_data(‘300274’)
#直接保存
df.to_excel(‘C:/Users/07010018/Desktop/tushare-1.1.6/exportdata/300274.xlsx’)

#设定数据位置(从第3行,第6列开始插入数据)
#df.to_excel(‘C:/Users/07010018/Desktop/tushare-1.1.6/exportdata/300274.xlsx’, startrow=2,startcol=5)
复制代码
保存为MySQL数据库

pandas提供了将数据便捷存入关系型数据库的方法,在新版的pandas中,主要是已sqlalchemy方式与数据建立连接,支持MySQL、Postgresql、Oracle、MS SQLServer、SQLite等主流数据库。本例以MySQL数据库为代表,展示将获取到的股票数据存入数据库的方法,其他类型数据库请参考sqlalchemy官网文档的create_engine部分。

常用参数说明:

name:表名,pandas会自动创建表结构
con:数据库连接,最好是用sqlalchemy创建engine的方式来替代con
flavor:数据库类型 {‘sqlite’, ‘mysql’}, 默认‘sqlite’,如果是engine此项可忽略
schema:指定数据库的schema,默认即可
if_exists:如果表名已存在的处理方式 {‘fail’, ‘replace’, ‘append’},默认‘fail’
index:将pandas的Index作为一列存入数据库,默认是True
index_label:Index的列名
chunksize:分批存入数据库,默认是None,即一次性全部写人数据库
dtype:设定columns在数据库里的数据类型,默认是None
  使用方法:

复制代码
from sqlalchemy import create_engine
import tushare as ts

df = ts.get_tick_data(‘300274′, date=’2018-03-27’)
engine = create_engine(‘mysql://user:passwd@127.0.0.1/db_name?charset=utf8’)

#存入数据库
df.to_sql(‘tick_data’,engine)

#追加数据到现有表
#df.to_sql(‘tick_data’,engine,if_exists=’append’)
复制代码

另外,数据也可以保存为HDF5格式,JSON格式,存入NoSQL数据库,这里就不在一一赘述,就简单介绍几种常用的。

股票接口

实时行情:http://stockhtm.finance.qq.com/sstock/ggcx/600103.shtml
http://qt.gtimg.cn/q=sh600103
http://web.sqt.gtimg.cn/q=sh600103
有当天总成交量 外盘内盘 买卖各5挡 成交量 成交额 市值 等信息。
分时成交 当天
http://stockhtm.finance.qq.com/sstock/quotpage/q/600103.htm#detail
http://stock.gtimg.cn/data/index.php?appn=detail&action=data&c=sh600103&p=0http://stock.gtimg.cn/data/index.php?appn=detail&action=data&c=sh600103&p=1
逐分钟 当天
http://stockhtm.finance.qq.com/sstock/ggcx/600103.shtml
http://web.ifzq.gtimg.cn/appstock/app/minute/query?code=sh600103http://web.ifzq.gtimg.cn/appstock/app/minute/query?_var=min_data_sh600103&code=sh600103&r=0.12174363108800135
逐分钟 当天 新浪投资易https://touzi.sina.com.cn/public/mystock
https://stock.sina.com.cn/stock/api/jsonp.php/var%20_sh6001032017=/TouziService.getStockMinuteFlow?symbol=sh600103&random=$rn
https://stock.sina.com.cn/stock/api/jsonp.php//TouziService.getStockMinuteFlow?symbol=sh600103&.replace(/\(newString\(|\)\);/g,”)
分价表 当天
http://stockhtm.finance.qq.com/sstock/quotpage/q/600103.htm#price
http://stock.gtimg.cn/data/index.php?appn=price&c=sh600103
日K 后复权http://web.ifzq.gtimg.cn/appstock/app/fqkline/get?_var=kline_dayhfq¶m=sh600103,day,,,320,hfq&r=0.9860043111257255320
代表查询几天的历史数据 初步推断 1年为320 两年为640
http://web.ifzq.gtimg.cn/appstock/app/fqkline/get?_var=kline_dayhfq¶m=sh600103,day,1998-01-01,1998-12-31,320,hfq&r=0.444157593883574
http://web.ifzq.gtimg.cn/appstock/app/fqkline/get?_var=kline_dayhfq¶m=sh600103,day,1999-01-01,1999-12-31,320,hfq&r=0.7529798413161188
http://web.ifzq.gtimg.cn/appstock/app/fqkline/get?_var=kline_dayhfq2000¶m=sh600103,day,2000-01-01,2001-12-31,640,hfq&r=0.7360555452760309
日K线 百度股市通
https://gupiao.baidu.com/stock/sh600103.html
https://gupiao.baidu.com/api/stocks/stockdaybar?from=pc&os_ver=1&cuid=xxx&vv=100&format=json&stock_code=sh600103&step=3&start=&count=320&fq_type=front×tamp=1486464067730
https://gupiao.baidu.com/api/stocks/stockdaybar?from=pc&os_ver=1&cuid=xxx&vv=100&format=json&stock_code=sh600103&step=3&start=20150907&count=160&fq_type=no×tamp=1486463762308
修改count的值为最大,不带start即可。 fq_type为复权。分时成交Excel:
http://stock.gtimg.cn/data/index.php?appn=detail&action=download&c=sh600103&d=20170124
旧新都有http://market.finance.sina.com.cn/downxls.php?date=2017-01-24&symbol=sh600103
新的全现货黄金:
http://quote.fx678.com/symbo?exchName=WGJS&symbol=XAU
http://api.q.fx678.com/quotes.php?exchName=WGJS&symbol=XAU

httplib库与http.client库

python3把httplib改了名字,对应的库是http.client
https://docs.python.org/3.4/library/http.client.html
https://docs.python.org/2/library/httplib.html

21.12. http.client — HTTP protocol client

Source code: Lib/http/client.py


This module defines classes which implement the client side of the HTTP and HTTPS protocols. It is normally not used directly — the moduleurllib.request uses it to handle URLs that use HTTP and HTTPS.

See also

The Requests package is recommended for a higher-level http client interface.

Note

HTTPS support is only available if Python was compiled with SSL support (through the ssl module).

The module provides the following classes:

class http.client.HTTPConnection(hostport=None[timeout]source_address=None)
An HTTPConnection instance represents one transaction with an HTTP server. It should be instantiated passing it a host and optional port number. If no port number is passed, the port is extracted from the host string if it has the form host:port, else the default HTTP port (80) is used. If the optional timeout parameter is given, blocking operations (like connection attempts) will timeout after that many seconds (if it is not given, the global default timeout setting is used). The optional source_address parameter may be a tuple of a (host, port) to use as the source address the HTTP connection is made from.

For example, the following calls all create instances that connect to the server at the same host and port:

>>>

>>> h1 = http.client.HTTPConnection('www.python.org')
>>> h2 = http.client.HTTPConnection('www.python.org:80')
>>> h3 = http.client.HTTPConnection('www.python.org', 80)
>>> h4 = http.client.HTTPConnection('www.python.org', 80, timeout=10)

Changed in version 3.2: source_address was added.

Changed in version 3.4: The strict parameter was removed. HTTP 0.9-style “Simple Responses” are not longer supported.

class http.client.HTTPSConnection(hostport=Nonekey_file=Nonecert_file=None[timeout]source_address=None*context=Nonecheck_hostname=None)
A subclass of HTTPConnection that uses SSL for communication with secure servers. Default port is 443. If context is specified, it must be a ssl.SSLContext instance describing the various SSL options.

key_file and cert_file are deprecated, please use ssl.SSLContext.load_cert_chain() instead, or let ssl.create_default_context() select the system’s trusted CA certificates for you. The check_hostname parameter is also deprecated; the ssl.SSLContext.check_hostname attribute of context should be used instead.

Please read Security considerations for more information on best practices.

Changed in version 3.2: source_addresscontext and check_hostname were added.

Changed in version 3.2: This class now supports HTTPS virtual hosts if possible (that is, if ssl.HAS_SNI is true).

Changed in version 3.4: The strict parameter was removed. HTTP 0.9-style “Simple Responses” are no longer supported.

Changed in version 3.4.3: This class now performs all the necessary certificate and hostname checks by default. To revert to the previous, unverified, behavior ssl._create_unverified_context() can be passed to the context parameter.

class http.client.HTTPResponse(sockdebuglevel=0method=Noneurl=None)
Class whose instances are returned upon successful connection. Not instantiated directly by user.

Changed in version 3.4: The strict parameter was removed. HTTP 0.9 style “Simple Responses” are no longer supported.

The following exceptions are raised as appropriate:

exception http.client.HTTPException
The base class of the other exceptions in this module. It is a subclass of Exception.
exception http.client.NotConnected
A subclass of HTTPException.
exception http.client.InvalidURL
A subclass of HTTPException, raised if a port is given and is either non-numeric or empty.
exception http.client.UnknownProtocol
A subclass of HTTPException.
exception http.client.UnknownTransferEncoding
A subclass of HTTPException.
exception http.client.UnimplementedFileMode
A subclass of HTTPException.
exception http.client.IncompleteRead
A subclass of HTTPException.
exception http.client.ImproperConnectionState
A subclass of HTTPException.
exception http.client.CannotSendRequest
A subclass of ImproperConnectionState.
exception http.client.CannotSendHeader
A subclass of ImproperConnectionState.
exception http.client.ResponseNotReady
A subclass of ImproperConnectionState.
exception http.client.BadStatusLine
A subclass of HTTPException. Raised if a server responds with a HTTP status code that we don’t understand.
exception http.client.LineTooLong
A subclass of HTTPException. Raised if an excessively long line is received in the HTTP protocol from the server.

The constants defined in this module are:

http.client.HTTP_PORT
The default port for the HTTP protocol (always 80).
http.client.HTTPS_PORT
The default port for the HTTPS protocol (always 443).

and also the following constants for integer status codes:

Constant Value Definition
CONTINUE 100 HTTP/1.1, RFC 2616, Section 10.1.1
SWITCHING_PROTOCOLS 101 HTTP/1.1, RFC 2616, Section 10.1.2
PROCESSING 102 WEBDAV, RFC 2518, Section 10.1
OK 200 HTTP/1.1, RFC 2616, Section 10.2.1
CREATED 201 HTTP/1.1, RFC 2616, Section 10.2.2
ACCEPTED 202 HTTP/1.1, RFC 2616, Section 10.2.3
NON_AUTHORITATIVE_INFORMATION 203 HTTP/1.1, RFC 2616, Section 10.2.4
NO_CONTENT 204 HTTP/1.1, RFC 2616, Section 10.2.5
RESET_CONTENT 205 HTTP/1.1, RFC 2616, Section 10.2.6
PARTIAL_CONTENT 206 HTTP/1.1, RFC 2616, Section 10.2.7
MULTI_STATUS 207 WEBDAV RFC 2518, Section 10.2
IM_USED 226 Delta encoding in HTTP, RFC 3229, Section 10.4.1
MULTIPLE_CHOICES 300 HTTP/1.1, RFC 2616, Section 10.3.1
MOVED_PERMANENTLY 301 HTTP/1.1, RFC 2616, Section 10.3.2
FOUND 302 HTTP/1.1, RFC 2616, Section 10.3.3
SEE_OTHER 303 HTTP/1.1, RFC 2616, Section 10.3.4
NOT_MODIFIED 304 HTTP/1.1, RFC 2616, Section 10.3.5
USE_PROXY 305 HTTP/1.1, RFC 2616, Section 10.3.6
TEMPORARY_REDIRECT 307 HTTP/1.1, RFC 2616, Section 10.3.8
BAD_REQUEST 400 HTTP/1.1, RFC 2616, Section 10.4.1
UNAUTHORIZED 401 HTTP/1.1, RFC 2616, Section 10.4.2
PAYMENT_REQUIRED 402 HTTP/1.1, RFC 2616, Section 10.4.3
FORBIDDEN 403 HTTP/1.1, RFC 2616, Section 10.4.4
NOT_FOUND 404 HTTP/1.1, RFC 2616, Section 10.4.5
METHOD_NOT_ALLOWED 405 HTTP/1.1, RFC 2616, Section 10.4.6
NOT_ACCEPTABLE 406 HTTP/1.1, RFC 2616, Section 10.4.7
PROXY_AUTHENTICATION_REQUIRED 407 HTTP/1.1, RFC 2616, Section 10.4.8
REQUEST_TIMEOUT 408 HTTP/1.1, RFC 2616, Section 10.4.9
CONFLICT 409 HTTP/1.1, RFC 2616, Section 10.4.10
GONE 410 HTTP/1.1, RFC 2616, Section 10.4.11
LENGTH_REQUIRED 411 HTTP/1.1, RFC 2616, Section 10.4.12
PRECONDITION_FAILED 412 HTTP/1.1, RFC 2616, Section 10.4.13
REQUEST_ENTITY_TOO_LARGE 413 HTTP/1.1, RFC 2616, Section 10.4.14
REQUEST_URI_TOO_LONG 414 HTTP/1.1, RFC 2616, Section 10.4.15
UNSUPPORTED_MEDIA_TYPE 415 HTTP/1.1, RFC 2616, Section 10.4.16
REQUESTED_RANGE_NOT_SATISFIABLE 416 HTTP/1.1, RFC 2616, Section 10.4.17
EXPECTATION_FAILED 417 HTTP/1.1, RFC 2616, Section 10.4.18
UNPROCESSABLE_ENTITY 422 WEBDAV, RFC 2518, Section 10.3
LOCKED 423 WEBDAV RFC 2518, Section 10.4
FAILED_DEPENDENCY 424 WEBDAV, RFC 2518, Section 10.5
UPGRADE_REQUIRED 426 HTTP Upgrade to TLS, RFC 2817, Section 6
PRECONDITION_REQUIRED 428 Additional HTTP Status Codes, RFC 6585, Section 3
TOO_MANY_REQUESTS 429 Additional HTTP Status Codes, RFC 6585, Section 4
REQUEST_HEADER_FIELDS_TOO_LARGE 431 Additional HTTP Status Codes, RFC 6585, Section 5
INTERNAL_SERVER_ERROR 500 HTTP/1.1, RFC 2616, Section 10.5.1
NOT_IMPLEMENTED 501 HTTP/1.1, RFC 2616, Section 10.5.2
BAD_GATEWAY 502 HTTP/1.1 RFC 2616, Section 10.5.3
SERVICE_UNAVAILABLE 503 HTTP/1.1, RFC 2616, Section 10.5.4
GATEWAY_TIMEOUT 504 HTTP/1.1 RFC 2616, Section 10.5.5
HTTP_VERSION_NOT_SUPPORTED 505 HTTP/1.1, RFC 2616, Section 10.5.6
INSUFFICIENT_STORAGE 507 WEBDAV, RFC 2518, Section 10.6
NOT_EXTENDED 510 An HTTP Extension Framework, RFC 2774, Section 7
NETWORK_AUTHENTICATION_REQUIRED 511 Additional HTTP Status Codes, RFC 6585, Section 6

Changed in version 3.3: Added codes 428429431 and 511 from RFC 6585.

http.client.responses
This dictionary maps the HTTP 1.1 status codes to the W3C names.

Example: http.client.responses[http.client.NOT_FOUND] is 'Not Found'.

 

21.12.1. HTTPConnection Objects

HTTPConnection instances have the following methods:

HTTPConnection.request(methodurlbody=Noneheaders={})
This will send a request to the server using the HTTP request method method and the selector url.

If body is specified, the specified data is sent after the headers are finished. It may be a string, a bytes-like object, an open file object, or an iterable of bytes-like objects. If body is a string, it is encoded as ISO-8851-1, the default for HTTP. If it is a bytes-like object the bytes are sent as is. If it is a file object, the contents of the file is sent; this file object should support at least the read() method. If the file object has a mode attribute, the data returned by the read() method will be encoded as ISO-8851-1 unless the mode attribute contains the substring b, otherwise the data returned by read() is sent as is. If body is an iterable, the elements of the iterable are sent as is until the iterable is exhausted.

The headers argument should be a mapping of extra HTTP headers to send with the request.

If headers does not contain a Content-Length item, one is added automatically if possible. If body is None, the Content-Length header is set to 0 for methods that expect a body (PUTPOST, and PATCH). If body is a string or bytes object, the Content-Length header is set to its length. If body is a file object and it works to call fstat() on the result of its fileno() method, then the Content-Length header is set to the st_size reported by the fstat call. Otherwise no Content-Length header is added.

New in version 3.2: body can now be an iterable.

HTTPConnection.getresponse()
Should be called after a request is sent to get the response from the server. Returns an HTTPResponse instance.

Note

Note that you must have read the whole response before you can send a new request to the server.

HTTPConnection.set_debuglevel(level)
Set the debugging level. The default debug level is 0, meaning no debugging output is printed. Any value greater than 0 will cause all currently defined debug output to be printed to stdout. The debuglevel is passed to any new HTTPResponse objects that are created.

New in version 3.1.

HTTPConnection.set_tunnel(hostport=Noneheaders=None)
Set the host and the port for HTTP Connect Tunnelling. This allows running the connection through a proxy server.

The host and port arguments specify the endpoint of the tunneled connection (i.e. the address included in the CONNECT request, not the address of the proxy server).

The headers argument should be a mapping of extra HTTP headers to send with the CONNECT request.

For example, to tunnel through a HTTPS proxy server running locally on port 8080, we would pass the address of the proxy to the HTTPSConnectionconstructor, and the address of the host that we eventually want to reach to the set_tunnel() method:

>>>

>>> import http.client
>>> conn = http.client.HTTPSConnection("localhost", 8080)
>>> conn.set_tunnel("www.python.org")
>>> conn.request("HEAD","/index.html")

New in version 3.2.

HTTPConnection.connect()
Connect to the server specified when the object was created.
HTTPConnection.close()
Close the connection to the server.

As an alternative to using the request() method described above, you can also send your request step by step, by using the four functions below.

HTTPConnection.putrequest(requestselectorskip_host=Falseskip_accept_encoding=False)
This should be the first call after the connection to the server has been made. It sends a line to the server consisting of the request string, the selectorstring, and the HTTP version (HTTP/1.1). To disable automatic sending of Host: or Accept-Encoding: headers (for example to accept additional content encodings), specify skip_host or skip_accept_encoding with non-False values.
HTTPConnection.putheader(headerargument[])
Send an RFC 822-style header to the server. It sends a line to the server consisting of the header, a colon and a space, and the first argument. If more arguments are given, continuation lines are sent, each consisting of a tab and an argument.
HTTPConnection.endheaders(message_body=None)
Send a blank line to the server, signalling the end of the headers. The optional message_body argument can be used to pass a message body associated with the request. The message body will be sent in the same packet as the message headers if it is string, otherwise it is sent in a separate packet.
HTTPConnection.send(data)
Send data to the server. This should be used directly only after the endheaders() method has been called and before getresponse() is called.

 

21.12.2. HTTPResponse Objects

An HTTPResponse instance wraps the HTTP response from the server. It provides access to the request headers and the entity body. The response is an iterable object and can be used in a with statement.

HTTPResponse.read([amt])
Reads and returns the response body, or up to the next amt bytes.
HTTPResponse.readinto(b)
Reads up to the next len(b) bytes of the response body into the buffer b. Returns the number of bytes read.

New in version 3.3.

HTTPResponse.getheader(namedefault=None)
Return the value of the header name, or default if there is no header matching name. If there is more than one header with the name name, return all of the values joined by ‘, ‘. If ‘default’ is any iterable other than a single string, its elements are similarly returned joined by commas.
HTTPResponse.getheaders()
Return a list of (header, value) tuples.
HTTPResponse.fileno()
Return the fileno of the underlying socket.
HTTPResponse.msg
http.client.HTTPMessage instance containing the response headers. http.client.HTTPMessage is a subclass of email.message.Message.
HTTPResponse.version
HTTP protocol version used by server. 10 for HTTP/1.0, 11 for HTTP/1.1.
HTTPResponse.status
Status code returned by server.
HTTPResponse.reason
Reason phrase returned by server.
HTTPResponse.debuglevel
A debugging hook. If debuglevel is greater than zero, messages will be printed to stdout as the response is read and parsed.
HTTPResponse.closed
Is True if the stream is closed.

21.12.3. Examples

Here is an example session that uses the GET method:

>>>

>>> import http.client
>>> conn = http.client.HTTPSConnection("www.python.org")
>>> conn.request("GET", "/")
>>> r1 = conn.getresponse()
>>> print(r1.status, r1.reason)
200 OK
>>> data1 = r1.read()  # This will return entire content.
>>> # The following example demonstrates reading data in chunks.
>>> conn.request("GET", "/")
>>> r1 = conn.getresponse()
>>> while not r1.closed:
...     print(r1.read(200)) # 200 bytes
b'<!doctype html>\n<!--[if"...
...
>>> # Example of an invalid request
>>> conn.request("GET", "/parrot.spam")
>>> r2 = conn.getresponse()
>>> print(r2.status, r2.reason)
404 Not Found
>>> data2 = r2.read()
>>> conn.close()

Here is an example session that uses the HEAD method. Note that the HEAD method never returns any data.

>>>

>>> import http.client
>>> conn = http.client.HTTPSConnection("www.python.org")
>>> conn.request("HEAD", "/")
>>> res = conn.getresponse()
>>> print(res.status, res.reason)
200 OK
>>> data = res.read()
>>> print(len(data))
0
>>> data == b''
True

Here is an example session that shows how to POST requests:

>>>

>>> import http.client, urllib.parse
>>> params = urllib.parse.urlencode({'@number': 12524, '@type': 'issue', '@action': 'show'})
>>> headers = {"Content-type": "application/x-www-form-urlencoded",
...            "Accept": "text/plain"}
>>> conn = http.client.HTTPConnection("bugs.python.org")
>>> conn.request("POST", "", params, headers)
>>> response = conn.getresponse()
>>> print(response.status, response.reason)
302 Found
>>> data = response.read()
>>> data
b'Redirecting to <a href="http://bugs.python.org/issue12524">http://bugs.python.org/issue12524</a>'
>>> conn.close()

Client side HTTP PUT requests are very similar to POST requests. The difference lies only the server side where HTTP server will allow resources to be created via PUT request. It should be noted that custom HTTP methods +are also handled in urllib.request.Request by sending the appropriate +method attribute.Here is an example session that shows how to do PUT request using http.client:

>>>

>>> # This creates an HTTP message
>>> # with the content of BODY as the enclosed representation
>>> # for the resource http://localhost:8080/file
...
>>> import http.client
>>> BODY = "***filecontents***"
>>> conn = http.client.HTTPConnection("localhost", 8080)
>>> conn.request("PUT", "/file", BODY)
>>> response = conn.getresponse()
>>> print(response.status, response.reason)
200, OK

 

21.12.4. HTTPMessage Objects

An http.client.HTTPMessage instance holds the headers from an HTTP response. It is implemented using the email.message.Message class.