pytorch+lstm实现的pos示例


Posted in Python onJanuary 14, 2020

学了几天终于大概明白pytorch怎么用了

这个是直接搬运的官方文档的代码

之后会自己试着实现其他nlp的任务

# Author: Robert Guthrie

import torch
import torch.autograd as autograd
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim

torch.manual_seed(1)


lstm = nn.LSTM(3, 3) # Input dim is 3, output dim is 3
inputs = [autograd.Variable(torch.randn((1, 3)))
     for _ in range(5)] # make a sequence of length 5

# initialize the hidden state.
hidden = (autograd.Variable(torch.randn(1, 1, 3)),
     autograd.Variable(torch.randn((1, 1, 3))))
for i in inputs:
  # Step through the sequence one element at a time.
  # after each step, hidden contains the hidden state.
  out, hidden = lstm(i.view(1, 1, -1), hidden)

# alternatively, we can do the entire sequence all at once.
# the first value returned by LSTM is all of the hidden states throughout
# the sequence. the second is just the most recent hidden state
# (compare the last slice of "out" with "hidden" below, they are the same)
# The reason for this is that:
# "out" will give you access to all hidden states in the sequence
# "hidden" will allow you to continue the sequence and backpropagate,
# by passing it as an argument to the lstm at a later time
# Add the extra 2nd dimension
inputs = torch.cat(inputs).view(len(inputs), 1, -1)
hidden = (autograd.Variable(torch.randn(1, 1, 3)), autograd.Variable(
  torch.randn((1, 1, 3)))) # clean out hidden state
out, hidden = lstm(inputs, hidden)
#print(out)
#print(hidden)

#准备数据
def prepare_sequence(seq, to_ix):
  idxs = [to_ix[w] for w in seq]
  tensor = torch.LongTensor(idxs)
  return autograd.Variable(tensor)

training_data = [
  ("The dog ate the apple".split(), ["DET", "NN", "V", "DET", "NN"]),
  ("Everybody read that book".split(), ["NN", "V", "DET", "NN"])
]
word_to_ix = {}
for sent, tags in training_data:
  for word in sent:
    if word not in word_to_ix:
      word_to_ix[word] = len(word_to_ix)
print(word_to_ix)
tag_to_ix = {"DET": 0, "NN": 1, "V": 2}

# These will usually be more like 32 or 64 dimensional.
# We will keep them small, so we can see how the weights change as we train.
EMBEDDING_DIM = 6
HIDDEN_DIM = 6

#继承自nn.module
class LSTMTagger(nn.Module):

  def __init__(self, embedding_dim, hidden_dim, vocab_size, tagset_size):
    super(LSTMTagger, self).__init__()
    self.hidden_dim = hidden_dim

    #一个单词数量到embedding维数的矩阵
    self.word_embeddings = nn.Embedding(vocab_size, embedding_dim)

    #传入两个维度参数
    # The LSTM takes word embeddings as inputs, and outputs hidden states
    # with dimensionality hidden_dim.
    self.lstm = nn.LSTM(embedding_dim, hidden_dim)

    #线性layer从隐藏状态空间映射到tag便签
    # The linear layer that maps from hidden state space to tag space
    self.hidden2tag = nn.Linear(hidden_dim, tagset_size)
    self.hidden = self.init_hidden()

  def init_hidden(self):
    # Before we've done anything, we dont have any hidden state.
    # Refer to the Pytorch documentation to see exactly
    # why they have this dimensionality.
    # The axes semantics are (num_layers, minibatch_size, hidden_dim)
    return (autograd.Variable(torch.zeros(1, 1, self.hidden_dim)),
        autograd.Variable(torch.zeros(1, 1, self.hidden_dim)))

  def forward(self, sentence):
    embeds = self.word_embeddings(sentence)
    lstm_out, self.hidden = self.lstm(embeds.view(len(sentence), 1, -1), self.hidden)
    tag_space = self.hidden2tag(lstm_out.view(len(sentence), -1))
    tag_scores = F.log_softmax(tag_space)
    return tag_scores

#embedding维度,hidden维度,词语数量,标签数量
model = LSTMTagger(EMBEDDING_DIM, HIDDEN_DIM, len(word_to_ix), len(tag_to_ix))

#optim中存了各种优化算法
loss_function = nn.NLLLoss()
optimizer = optim.SGD(model.parameters(), lr=0.1)

# See what the scores are before training
# Note that element i,j of the output is the score for tag j for word i.
inputs = prepare_sequence(training_data[0][0], word_to_ix)
tag_scores = model(inputs)
print(tag_scores)

for epoch in range(300): # again, normally you would NOT do 300 epochs, it is toy data
  for sentence, tags in training_data:
    # Step 1. Remember that Pytorch accumulates gradients.
    # We need to clear them out before each instance
    model.zero_grad()

    # Also, we need to clear out the hidden state of the LSTM,
    # detaching it from its history on the last instance.
    model.hidden = model.init_hidden()

    # Step 2. Get our inputs ready for the network, that is, turn them into
    # Variables of word indices.
    sentence_in = prepare_sequence(sentence, word_to_ix)
    targets = prepare_sequence(tags, tag_to_ix)

    # Step 3. Run our forward pass.
    tag_scores = model(sentence_in)

    # Step 4. Compute the loss, gradients, and update the parameters by
    # calling optimizer.step()
    loss = loss_function(tag_scores, targets)
    loss.backward()
    optimizer.step()

# See what the scores are after training
inputs = prepare_sequence(training_data[0][0], word_to_ix)
tag_scores = model(inputs)
# The sentence is "the dog ate the apple". i,j corresponds to score for tag j
# for word i. The predicted tag is the maximum scoring tag.
# Here, we can see the predicted sequence below is 0 1 2 0 1
# since 0 is index of the maximum value of row 1,
# 1 is the index of maximum value of row 2, etc.
# Which is DET NOUN VERB DET NOUN, the correct sequence!
print(tag_scores)

以上这篇pytorch+lstm实现的pos示例就是小编分享给大家的全部内容了,希望能给大家一个参考,也希望大家多多支持三水点靠木。

Python 相关文章推荐
python时间整形转标准格式的示例分享
Feb 14 Python
python基础教程之Filter使用方法
Jan 17 Python
Python网络爬虫中的同步与异步示例详解
Feb 03 Python
谈谈python中GUI的选择
Mar 01 Python
使用python判断你是青少年还是老年人
Nov 29 Python
pandas 层次化索引的实现方法
Jul 06 Python
Python代码生成视频的缩略图的实例讲解
Dec 22 Python
python 统计文件中的字符串数目示例
Dec 24 Python
快速查找Python安装路径方法
Feb 06 Python
Python程序慢的重要原因
Sep 04 Python
Pytest单元测试框架如何实现参数化
Sep 05 Python
python读取mat文件生成h5文件的实现
Jul 15 Python
Python中sorted()排序与字母大小写的问题
Jan 14 #Python
Pytorch实现LSTM和GRU示例
Jan 14 #Python
Python生成词云的实现代码
Jan 14 #Python
pytorch-RNN进行回归曲线预测方式
Jan 14 #Python
利用pytorch实现对CIFAR-10数据集的分类
Jan 14 #Python
pytorch下使用LSTM神经网络写诗实例
Jan 14 #Python
python使用openCV遍历文件夹里所有视频文件并保存成图片
Jan 14 #Python
You might like
实例(Smarty+FCKeditor新闻系统)
2007/01/02 PHP
PHP生成不重复标识符的方法
2014/11/21 PHP
经典的解除许多网站无法复制文字的绝招
2006/12/31 Javascript
js或css文件后面跟参数的原因说明
2010/01/09 Javascript
js 复制或插入Html的实现方法小结
2010/05/19 Javascript
JavaScript中将一个值转换为字符串的方法分析[译]
2012/09/21 Javascript
Javascript判断对象是否相等实现代码
2013/03/18 Javascript
jCallout 轻松实现气泡提示功能
2013/09/22 Javascript
Javascript Web Slider 焦点图示例源码
2013/10/10 Javascript
只需一行代码,轻松实现一个在线编辑器
2013/11/12 Javascript
借助javascript代码判断网页是静态还是伪静态
2014/05/05 Javascript
Jquery实现地铁线路指示灯提示牌效果的方法
2015/03/02 Javascript
Node.js实现Excel转JSON
2015/04/24 Javascript
BootStrap智能表单实战系列(八)表单配置json详解
2016/06/13 Javascript
微信小程序 数据绑定详解及实例
2016/10/25 Javascript
浅谈JavaScript的闭包函数
2016/12/08 Javascript
ES6生成器用法实例分析
2017/04/10 Javascript
js实现简易聊天对话框
2017/08/17 Javascript
Angularjs 手写日历的实现代码(不用插件)
2017/10/18 Javascript
vue 项目打包通过命令修改 vue-router 模式 修改 API 接口前缀
2018/06/13 Javascript
layui 实现自动选择radio单选框(checked)的方法
2019/09/03 Javascript
layer.open回调获取弹出层参数的实现方法
2019/09/10 Javascript
python监控网站运行异常并发送邮件的方法
2015/03/13 Python
Python中的模块和包概念介绍
2015/04/13 Python
玩转python爬虫之爬取糗事百科段子
2016/02/17 Python
Python全局变量与局部变量区别及用法分析
2018/09/03 Python
提升Python程序性能的7个习惯
2019/04/14 Python
PHP统计代码行数的小代码
2019/09/19 Python
python栈的基本定义与使用方法示例【初始化、赋值、入栈、出栈等】
2019/10/24 Python
Django中密码的加密、验密、解密操作
2019/12/19 Python
PyTorch实现更新部分网络,其他不更新
2019/12/31 Python
浅谈pytorch卷积核大小的设置对全连接神经元的影响
2020/01/10 Python
会议活动邀请函
2014/01/27 职场文书
党性观念心得体会
2014/09/03 职场文书
2015年国庆放假通知范文
2015/08/18 职场文书
小学校园广播稿
2015/08/18 职场文书