PyTorch torch.nn.SiLU 函数
PyTorch torch.nn 参考手册torch.nn.SiLU 是 PyTorch 中的 Sigmoid Linear Unit 激活函数，也称为 Swish。
它具有自门控特性，比 ReLU 更平滑，在某些任务上表现更好。
函数定义
torch.nn.SiLU(inplace=False)
数学原理
SiLU(x) = x * sigmoid(x)
使用示例
示例 1: 基本用法
实例
import torch
import torch.nn as nn
silu = nn.SiLU()
x = torch.tensor([-2.0, -1.0, 0.0, 1.0, 2.0])
output = silu(x)
print("输入:", x.tolist())
print("输出:", output.tolist())
import torch.nn as nn
silu = nn.SiLU()
x = torch.tensor([-2.0, -1.0, 0.0, 1.0, 2.0])
output = silu(x)
print("输入:", x.tolist())
print("输出:", output.tolist())
示例 2: 对比 ReLU
实例
import torch
import torch.nn as nn
import numpy as np
x = np.linspace(-3, 3, 7)
x_tensor = torch.tensor(x)
print("x SiLU ReLU")
print("-" * 30)
for xi in x_tensor:
print(f"{xi.item():5.1f} {nn.SiLU()(xi.unsqueeze(0)).item():9.4f} {nn.ReLU()(xi.unsqueeze(0)).item():9.4f}")
import torch.nn as nn
import numpy as np
x = np.linspace(-3, 3, 7)
x_tensor = torch.tensor(x)
print("x SiLU ReLU")
print("-" * 30)
for xi in x_tensor:
print(f"{xi.item():5.1f} {nn.SiLU()(xi.unsqueeze(0)).item():9.4f} {nn.ReLU()(xi.unsqueeze(0)).item():9.4f}")
示例 3: 在 MobileNet 中使用
实例
import torch
import torch.nn as nn
# MobileNetV3 使用 SiLU
model = nn.Sequential(
nn.Conv2d(3, 32, 3, stride=2, padding=1),
nn.BatchNorm2d(32),
nn.SiLU(),
nn.Conv2d(32, 64, 3, padding=1),
nn.BatchNorm2d(64),
nn.SiLU()
)
x = torch.randn(1, 3, 224, 224)
output = model(x)
print("输入:", x.shape, "-> 输出:", output.shape)
import torch.nn as nn
# MobileNetV3 使用 SiLU
model = nn.Sequential(
nn.Conv2d(3, 32, 3, stride=2, padding=1),
nn.BatchNorm2d(32),
nn.SiLU(),
nn.Conv2d(32, 64, 3, padding=1),
nn.BatchNorm2d(64),
nn.SiLU()
)
x = torch.randn(1, 3, 224, 224)
output = model(x)
print("输入:", x.shape, "-> 输出:", output.shape)
使用场景
- MobileNet: MobileNetV3
- EfficientNet
- 平滑激活: 需要门控特性
注意：SiLU 计算比 ReLU 复杂，但性能通常更好。