pytorch+lstm实现的pos示例


Posted in Python onJanuary 14, 2020

学了几天终于大概明白pytorch怎么用了

这个是直接搬运的官方文档的代码

之后会自己试着实现其他nlp的任务

# Author: Robert Guthrie

import torch
import torch.autograd as autograd
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim

torch.manual_seed(1)


lstm = nn.LSTM(3, 3) # Input dim is 3, output dim is 3
inputs = [autograd.Variable(torch.randn((1, 3)))
     for _ in range(5)] # make a sequence of length 5

# initialize the hidden state.
hidden = (autograd.Variable(torch.randn(1, 1, 3)),
     autograd.Variable(torch.randn((1, 1, 3))))
for i in inputs:
  # Step through the sequence one element at a time.
  # after each step, hidden contains the hidden state.
  out, hidden = lstm(i.view(1, 1, -1), hidden)

# alternatively, we can do the entire sequence all at once.
# the first value returned by LSTM is all of the hidden states throughout
# the sequence. the second is just the most recent hidden state
# (compare the last slice of "out" with "hidden" below, they are the same)
# The reason for this is that:
# "out" will give you access to all hidden states in the sequence
# "hidden" will allow you to continue the sequence and backpropagate,
# by passing it as an argument to the lstm at a later time
# Add the extra 2nd dimension
inputs = torch.cat(inputs).view(len(inputs), 1, -1)
hidden = (autograd.Variable(torch.randn(1, 1, 3)), autograd.Variable(
  torch.randn((1, 1, 3)))) # clean out hidden state
out, hidden = lstm(inputs, hidden)
#print(out)
#print(hidden)

#准备数据
def prepare_sequence(seq, to_ix):
  idxs = [to_ix[w] for w in seq]
  tensor = torch.LongTensor(idxs)
  return autograd.Variable(tensor)

training_data = [
  ("The dog ate the apple".split(), ["DET", "NN", "V", "DET", "NN"]),
  ("Everybody read that book".split(), ["NN", "V", "DET", "NN"])
]
word_to_ix = {}
for sent, tags in training_data:
  for word in sent:
    if word not in word_to_ix:
      word_to_ix[word] = len(word_to_ix)
print(word_to_ix)
tag_to_ix = {"DET": 0, "NN": 1, "V": 2}

# These will usually be more like 32 or 64 dimensional.
# We will keep them small, so we can see how the weights change as we train.
EMBEDDING_DIM = 6
HIDDEN_DIM = 6

#继承自nn.module
class LSTMTagger(nn.Module):

  def __init__(self, embedding_dim, hidden_dim, vocab_size, tagset_size):
    super(LSTMTagger, self).__init__()
    self.hidden_dim = hidden_dim

    #一个单词数量到embedding维数的矩阵
    self.word_embeddings = nn.Embedding(vocab_size, embedding_dim)

    #传入两个维度参数
    # The LSTM takes word embeddings as inputs, and outputs hidden states
    # with dimensionality hidden_dim.
    self.lstm = nn.LSTM(embedding_dim, hidden_dim)

    #线性layer从隐藏状态空间映射到tag便签
    # The linear layer that maps from hidden state space to tag space
    self.hidden2tag = nn.Linear(hidden_dim, tagset_size)
    self.hidden = self.init_hidden()

  def init_hidden(self):
    # Before we've done anything, we dont have any hidden state.
    # Refer to the Pytorch documentation to see exactly
    # why they have this dimensionality.
    # The axes semantics are (num_layers, minibatch_size, hidden_dim)
    return (autograd.Variable(torch.zeros(1, 1, self.hidden_dim)),
        autograd.Variable(torch.zeros(1, 1, self.hidden_dim)))

  def forward(self, sentence):
    embeds = self.word_embeddings(sentence)
    lstm_out, self.hidden = self.lstm(embeds.view(len(sentence), 1, -1), self.hidden)
    tag_space = self.hidden2tag(lstm_out.view(len(sentence), -1))
    tag_scores = F.log_softmax(tag_space)
    return tag_scores

#embedding维度,hidden维度,词语数量,标签数量
model = LSTMTagger(EMBEDDING_DIM, HIDDEN_DIM, len(word_to_ix), len(tag_to_ix))

#optim中存了各种优化算法
loss_function = nn.NLLLoss()
optimizer = optim.SGD(model.parameters(), lr=0.1)

# See what the scores are before training
# Note that element i,j of the output is the score for tag j for word i.
inputs = prepare_sequence(training_data[0][0], word_to_ix)
tag_scores = model(inputs)
print(tag_scores)

for epoch in range(300): # again, normally you would NOT do 300 epochs, it is toy data
  for sentence, tags in training_data:
    # Step 1. Remember that Pytorch accumulates gradients.
    # We need to clear them out before each instance
    model.zero_grad()

    # Also, we need to clear out the hidden state of the LSTM,
    # detaching it from its history on the last instance.
    model.hidden = model.init_hidden()

    # Step 2. Get our inputs ready for the network, that is, turn them into
    # Variables of word indices.
    sentence_in = prepare_sequence(sentence, word_to_ix)
    targets = prepare_sequence(tags, tag_to_ix)

    # Step 3. Run our forward pass.
    tag_scores = model(sentence_in)

    # Step 4. Compute the loss, gradients, and update the parameters by
    # calling optimizer.step()
    loss = loss_function(tag_scores, targets)
    loss.backward()
    optimizer.step()

# See what the scores are after training
inputs = prepare_sequence(training_data[0][0], word_to_ix)
tag_scores = model(inputs)
# The sentence is "the dog ate the apple". i,j corresponds to score for tag j
# for word i. The predicted tag is the maximum scoring tag.
# Here, we can see the predicted sequence below is 0 1 2 0 1
# since 0 is index of the maximum value of row 1,
# 1 is the index of maximum value of row 2, etc.
# Which is DET NOUN VERB DET NOUN, the correct sequence!
print(tag_scores)

以上这篇pytorch+lstm实现的pos示例就是小编分享给大家的全部内容了,希望能给大家一个参考,也希望大家多多支持三水点靠木。

Python 相关文章推荐
python让图片按照exif信息里的创建时间进行排序的方法
Mar 16 Python
python基础教程之匿名函数lambda
Jan 17 Python
Python实现的密码强度检测器示例
Aug 23 Python
Python工程师面试必备25条知识点
Jan 17 Python
python爬取个性签名的方法
Jun 17 Python
python实现LBP方法提取图像纹理特征实现分类的步骤
Jul 11 Python
python 读取修改pcap包的例子
Jul 23 Python
使用Python求解带约束的最优化问题详解
Feb 11 Python
Python交互环境下打印和输入函数的实例内容
Feb 16 Python
python实现将列表中各个值快速赋值给多个变量
Apr 02 Python
python 实现超级玛丽游戏
Nov 25 Python
将不规则的Python多维数组拉平到一维的方法实现
Jan 11 Python
Python中sorted()排序与字母大小写的问题
Jan 14 #Python
Pytorch实现LSTM和GRU示例
Jan 14 #Python
Python生成词云的实现代码
Jan 14 #Python
pytorch-RNN进行回归曲线预测方式
Jan 14 #Python
利用pytorch实现对CIFAR-10数据集的分类
Jan 14 #Python
pytorch下使用LSTM神经网络写诗实例
Jan 14 #Python
python使用openCV遍历文件夹里所有视频文件并保存成图片
Jan 14 #Python
You might like
一个显示天气预报的程序
2006/10/09 PHP
人大复印资料处理程序_补充篇
2006/10/09 PHP
组合算法的PHP解答方法
2012/02/04 PHP
PHP防止刷新重复提交页面的示例代码
2015/11/11 PHP
Laravel 5.3 学习笔记之 错误&日志
2016/08/28 PHP
在JavaScript中实现类的方式探讨
2013/08/28 Javascript
JQuery的ready函数与JS的onload的区别详解
2013/11/21 Javascript
JS中判断null、undefined与NaN的方法
2014/03/24 Javascript
jQuery实现的网页竖向菜单效果代码
2015/08/26 Javascript
悬浮广告方法日常收集整理
2016/03/18 Javascript
js和jq使用submit方法无法提交表单的快速解决方法
2016/05/17 Javascript
Spring MVC中Ajax实现二级联动的简单实例
2016/07/06 Javascript
快速解决js开发下拉框中blur与click冲突
2016/10/10 Javascript
Web纯前端“旭日图”实现元素周期表
2017/03/10 Javascript
封装运动框架实战左右与上下滑动的焦点轮播图(实例)
2017/10/17 Javascript
Vue $emit $refs子父组件间方法的调用实例
2018/09/12 Javascript
从零到一详聊创建Vue工程及遇到的常见问题
2019/04/25 Javascript
用Vue.js在浏览器中实现裁剪图像功能
2019/06/18 Javascript
vuex入门最详细整理
2020/03/04 Javascript
[07:06]2018DOTA2国际邀请赛寻真——卫冕冠军Team Liquid
2018/08/10 DOTA
Python获取Linux系统下的本机IP地址代码分享
2014/11/07 Python
Python标准异常和异常处理详解
2015/02/02 Python
Python实现telnet服务器的方法
2015/07/10 Python
详解python中requirements.txt的一切
2017/03/03 Python
详谈Python2.6和Python3.0中对除法操作的异同
2017/04/28 Python
python实现快速排序的示例(二分法思想)
2018/03/12 Python
python GUI库图形界面开发之PyQt5日期时间控件QDateTimeEdit详细使用方法与实例
2020/02/27 Python
Django中使用Json返回数据的实现方法
2020/06/03 Python
详解Pymongo常用查询方法总结
2021/01/29 Python
英国高档时尚男装购物网站:MR PORTER
2016/08/09 全球购物
世界上第一个水枕头:Mediflow
2018/12/06 全球购物
大学毕业生求职自荐信
2014/02/20 职场文书
安全责任书范本
2014/04/15 职场文书
公司联欢会策划方案
2014/05/19 职场文书
医院领导班子整改方案
2014/10/01 职场文书
MYSQL优化之数据表碎片整理详解
2022/04/03 MySQL