PyTorch之list、ndarray、tensor数据类型相互转换

发布于：2024-05-16 ⋅ 阅读:(39) ⋅ 点赞:(0)

温故而知新，可以为师矣！

一、参考资料

python中list、numpy、torch.tensor之间的相互转换

二、常用操作

list 转 numpy

ndarray = np.array(list)

import numpy as np

a_list = [[j for j in range(5)] for i in range(3)]
a_ndarray = np.array(a_list)

print(f'a_list = {a_list}, type of a_list: {type(a_list)}')
print(f'a_ndarray = {a_ndarray}, type of a_ndarray: {type(a_ndarray)}')

输出结果

a_list = [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], type of a_list: <class 'list'>
a_ndarray = [[0 1 2 3 4]
 [0 1 2 3 4]
 [0 1 2 3 4]], type of a_ndarray: <class 'numpy.ndarray'>

list 转 torch.Tensor

如何将装有tensor的多维list转化为torch.Tensor类型

普通 list 转 torch.Tensor

tensor=torch.Tensor(list)

# 普通list转tensor
a_list = [[j for j in range(5)] for i in range(3)]
A_tensor = torch.Tensor(a_list)

print(f'a_list = {a_list}, type of a_list: {type(a_list)}')
print(f'A_tensor = {A_tensor}, type of A_tensor: {type(A_tensor)}')

注意：将list中元素类型为int，转换为tensor后，类型转换为float，如果希望转换为int，则需要加上类型。

A_tensor = torch.Tensor(a_list)  # 默认为float
A_tensor = torch.IntTensor(a_list)  # 转为int

输出结果

a_list = [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], type of a_list: <class 'list'>
A_tensor = tensor([[0., 1., 2., 3., 4.],
        [0., 1., 2., 3., 4.],
        [0., 1., 2., 3., 4.]]), type of A_tensor: <class 'torch.Tensor'>

list包含多维tensor

# list包含tensor，使用torch.Tensor会报错
a = torch.rand((2, 3))
a_list = [a for _ in range(3)]
A_tensor = torch.Tensor(a_list)

输出结果

raceback (most recent call last):
  File "/PATH/TO/demo.py", line 13, in <module>
    A = torch.Tensor(a_list)
ValueError: only one element tensors can be converted to Python scalars

解决办法

如果该方法无法解决该问题，请参考下文的FAQ。

# 在cpu上
A_tensor= torch.tensor([item.detach().numpy() for item in a_list])

# 在gpu上
A_tensor= torch.tensor([item.cpu().detach().numpy() for item in a_list]).cuda()

注意：因为 gpu上的 tensor 不能直接转为 numpy，需要先在 cpu 上完成操作，再回到 gpu 上。

numpy 转 list

list = ndarray.tolist()

import numpy as np


a_list = [[j for j in range(5)] for i in range(3)]
a_ndarray = np.array(a_list)  # ndarray 转为 ndarray
A_list = a_ndarray.tolist()  # ndarray 转为 list

print(f'a_list = {a_list}, type of a_list: {type(a_list)}')
print(f'a_ndarray = {a_ndarray}, type of a_ndarray: {type(a_ndarray)}')
print(f'A_list = {A_list}, type of A_list: {type(A_list)}')

输出结果

a_list = [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], type of a_list: <class 'list'>
a_ndarray = [[0 1 2 3 4]
 [0 1 2 3 4]
 [0 1 2 3 4]], type of a_ndarray: <class 'numpy.ndarray'>
A_list = [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], type of A_list: <class 'list'>

numpy 转 torch.Tensor

tensor = torch.from_numpy(ndarray)

import torch
import numpy as np


a_list = [[j for j in range(5)] for i in range(3)]
a_ndarray = np.array(a_list)
a_tensor = torch.from_numpy(a_ndarray)

print(f'a_list = {a_list}, type of a_list: {type(a_list)}')
print(f'a_ndarray = {a_ndarray}, type of a_ndarray: {type(a_ndarray)}')
print(f'a_tensor = {a_tensor}, type of a_tensor: {type(a_tensor)}')

输出结果

a_list = [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], type of a_list: <class 'list'>
a_ndarray = [[0 1 2 3 4]
 [0 1 2 3 4]
 [0 1 2 3 4]], type of a_ndarray: <class 'numpy.ndarray'>
a_tensor = tensor([[0, 1, 2, 3, 4],
        [0, 1, 2, 3, 4],
        [0, 1, 2, 3, 4]]), type of a_tensor: <class 'torch.Tensor'>

torch.Tensor 转 numpy

# CPU
ndarray = tensor.numpy()

# GPU
ndarray = tensor.cpu().numpy()

注意：gpu上的tensor不能直接转为numpy，须要先在 cpu 上完成操做，再回到 gpu 上。

import torch


a_list = [[j for j in range(5)] for i in range(3)]
# list转tensor
A_tensor = torch.Tensor(a_list)

# CPU
A_ndarray = A_tensor.numpy()

print(f'a_list = {a_list}, type of a_list: {type(a_list)}')
print(f'A_tensor = {A_tensor}, type of A_tensor: {type(A_tensor)}')
print(f'A_ndarray = {A_ndarray}, type of A_ndarray: {type(A_ndarray)}')

输出结果

a_list = [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], type of a_list: <class 'list'>
A_tensor = tensor([[0., 1., 2., 3., 4.],
        [0., 1., 2., 3., 4.],
        [0., 1., 2., 3., 4.]]), type of A_tensor: <class 'torch.Tensor'>
A_ndarray = [[0. 1. 2. 3. 4.]
 [0. 1. 2. 3. 4.]
 [0. 1. 2. 3. 4.]], type of A_ndarray: <class 'numpy.ndarray'>

torch.Tensor 转 list

tensor先转numpy，后转list。

list = tensor.numpy().tolist()

import torch


a_list = [[j for j in range(5)] for i in range(3)]
# list转tensor
A_tensor = torch.Tensor(a_list)
# tensor先转numpy，再转list
A_list = A_tensor.numpy().tolist()

print(f'a_list = {a_list}, type of a_list: {type(a_list)}')
print(f'A_tensor = {A_tensor}, type of A_tensor: {type(A_tensor)}')
print(f'A_list = {A_list}, type of A_list: {type(A_list)}')

输出结果

a_list = [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], type of a_list: <class 'list'>
A_tensor = tensor([[0., 1., 2., 3., 4.],
        [0., 1., 2., 3., 4.],
        [0., 1., 2., 3., 4.]]), type of A_tensor: <class 'torch.Tensor'>
A_list = [[0.0, 1.0, 2.0, 3.0, 4.0], [0.0, 1.0, 2.0, 3.0, 4.0], [0.0, 1.0, 2.0, 3.0, 4.0]], type of A_list: <class 'list'>

三、FAQ

Q：ValueError: only one element tensors can be converted to Python scalars

Pytorch: list, numpy. Tensor 格式转化 （附 only one element tensors can be converted to Python scalars 解决）

ValueError:only one element tensors can be converted to Python scalars解决办法

错误原因：list包含多维tensor，导致类型转换错误。有以下两种解决方法。

方法一：torch.stack

通过torch.stack将包含tensor的多维list转换成tensor。
torch.stack要求两个输入的shape完全相同。

b_tensor = torch.rand((2, 3))
b_list = [b_tensor for _ in range(3)]
B_tensor = torch.stack(b_list)

print(f'b_tensor = {b_tensor}, type of b: {type(b_tensor)}')
print(f'b_list = {b_list}, type of b_list: {type(b_list)}')
print(f'B_tensor = {B_tensor}, type of B_tensor: {type(B_tensor)}, shape of B_tensor: {B_tensor.shape}')

b_tensor = tensor([[0.7443, 0.3041, 0.9545],
        [0.3092, 0.2747, 0.6717]]), type of b: <class 'torch.Tensor'>
b_list = [tensor([[0.7443, 0.3041, 0.9545],
        [0.3092, 0.2747, 0.6717]]), tensor([[0.7443, 0.3041, 0.9545],
        [0.3092, 0.2747, 0.6717]]), tensor([[0.7443, 0.3041, 0.9545],
        [0.3092, 0.2747, 0.6717]])], type of b_list: <class 'list'>
B_tensor = tensor([[[0.7443, 0.3041, 0.9545],
         [0.3092, 0.2747, 0.6717]],

        [[0.7443, 0.3041, 0.9545],
         [0.3092, 0.2747, 0.6717]],

        [[0.7443, 0.3041, 0.9545],
         [0.3092, 0.2747, 0.6717]]]), type of B_tensor: <class 'torch.Tensor'>, shape of B_tensor: torch.Size([3, 2, 3])

方法二：torch.cat

通过torch.cat将包含tensor的多维list转换成tensor。

b_tensor = torch.rand((2, 3))
b_list = [b_tensor for _ in range(3)]
B_tensor = torch.cat(b_list, 0)

print(f'b_tensor = {b_tensor}, type of b: {type(b_tensor)}')
print(f'b_list = {b_list}, type of b_list: {type(b_list)}')
print(f'B_tensor = {B_tensor}, type of B_tensor: {type(B_tensor)}, shape of B_tensor: {B_tensor.shape}')

b_tensor = tensor([[0.4237, 0.4743, 0.5213],
        [0.0815, 0.6654, 0.8780]]), type of b: <class 'torch.Tensor'>
b_list = [tensor([[0.4237, 0.4743, 0.5213],
        [0.0815, 0.6654, 0.8780]]), tensor([[0.4237, 0.4743, 0.5213],
        [0.0815, 0.6654, 0.8780]]), tensor([[0.4237, 0.4743, 0.5213],
        [0.0815, 0.6654, 0.8780]])], type of b_list: <class 'list'>
B_tensor = tensor([[0.4237, 0.4743, 0.5213],
        [0.0815, 0.6654, 0.8780],
        [0.4237, 0.4743, 0.5213],
        [0.0815, 0.6654, 0.8780],
        [0.4237, 0.4743, 0.5213],
        [0.0815, 0.6654, 0.8780]]), type of B_tensor: <class 'torch.Tensor'>, shape of B_tensor: torch.Size([6, 3])

网站公告

今日签到

点亮在社区的每一天
去签到

热门文章

最新发布