[正版]神经网络与深度学习 邱锡鹏 著 书籍 计算机人工智能技术丛书 卷积神经网络 递归神经网络 计算机视觉自然语言

●序前言常用符号表第一部分 机器学习基础第1章 绪论31.1人工智能...............................41.1.1人工智能的发展历史....................51.1.2人工智能的流派.......................71.2机器学习...............................71.3表示学习...............................81.3.1局部表示和分布式表示...................91.3.2表示学习...........................111.4深度学习...............................111.4.1端到端学习..........................121.5神经网络...............................131.5.1人脑神经网络........................131.5.2人工神经网络........................141.5.3神经网络的发展历史....................151.6本书的知识体系...........................171.7常用的深度学习框架.........................181.8总结和深入阅读...........................20第2章 机器学习概述232.1基本概念...............................242.2机器学习的三个基本要素......................262.2.1模型..............................262.2.2学习准则...........................272.2.3优化算法...........................302.3机器学习的简单示例——线性回归.................332.3.1参数学习...........................342.4偏差-方差分解............................382.5机器学习算法的类型.........................412.6数据的特征表示...........................432.6.1传统的特征学习.......................442.6.2深度学习方法........................462.7评价指标...............................462.8理论和定理..............................492.8.1PAC学习理论........................492.8.2没有免费午餐定理......................502.8.3奥卡姆剃刀原理.......................502.8.4丑小鸭定理..........................512.8.5归纳偏置...........................512.9总结和深入阅读...........................51第3章 线性模型3.1线性判别函数和决策边界......................563.1.1二分类............................563.1.2多分类............................583.2Logistic回归.............................593.2.1参数学习...........................603.3Softmax回归.............................613.3.1参数学习...........................623.4感知器.................................643.4.1参数学习...........................643.4.2感知器的收敛性.......................663.4.3参数平均感知器.......................673.4.4扩展到多分类........................693.5支持向量机..............................713.5.1参数学习...........................733.5.2核函数............................743.5.3软间隔............................743.6损失函数对比.............................753.7总结和深入阅读...........................76第二部分 基础模型第4章 前馈神经网络814.1神经元.................................824.1.1Sigmoid型函数.......................834.1.2ReLU函数..........................864.1.3Swish函数..........................884.1.4GELU函数..........................894.1.5Maxout单元.........................894.2网络结构...............................904.2.1前馈网络...........................904.2.2记忆网络...........................904.2.3图网络............................904.3前馈神经网络.............................914.3.1通用近似定理........................934.3.2应用到机器学习.......................944.3.3参数学习...........................954.4反向传播算法.............................954.5自动梯度计算.............................984.5.1数值微分...........................994.5.2符号微分...........................994.5.3自动微分...........................1004.6优化问题...............................1034.6.1非凸优化问题........................1034.6.2梯度消失问题........................1044.7总结和深入阅读...........................104第5章 卷积神经网络1095.1卷积..................................1105.1.1卷积的定义..........................1105.1.2互相关............................1125.1.3卷积的变种..........................1135.1.4卷积的数学性质.......................1145.2卷积神经网络.............................1155.2.1用卷积来代替全连接....................1155.2.2卷积层............................1165.2.3汇聚层............................1185.2.4卷积网络的整体结构....................1195.3参数学习...............................1205.3.1卷积神经网络的反向传播算法...............1205.4几种典型的卷积神经网络......................1215.4.1LeNet-5............................1225.4.2AlexNet...........................1235.4.3Inception网络........................1255.4.4残差网络...........................1265.5其他卷积方式.............................1275.5.1转置卷积...........................1275.5.2空洞卷积...........................1295.6总结和深入阅读...........................130第6章 循环神经网络1336.1给网络增加记忆能力.........................1346.1.1延时神经网络........................1346.1.2有外部输入的非线性自回归模型..............1346.1.3循环神经网络........................1356.2简单循环网络.............................1356.2.1循环神经网络的计算能力..................1366.3应用到机器学习...........................1386.3.1序列到类别模式.......................1386.3.2同步的序列到序列模式...................1396.3.3异步的序列到序列模式...................1396.4参数学习...............................1406.4.1随时间反向传播算法....................1416.4.2实时循环学习算法......................1426.5长程依赖问题.............................1436.5.1改进方案...........................1446.6基于门控的循环神经网络......................1456.6.1长短期记忆网络.......................1456.6.2LSTM网络的各种变体...................1476.6.3门控循环单元网络......................1486.7深层循环神经网络..........................1496.7.1堆叠循环神经网络......................1506.7.2双向循环神经网络......................1506.8扩展到图结构.............................1516.8.1递归神经网络........................1516.8.2图神经网络..........................1526.9总结和深入阅读...........................153第7章 网络优化与正则化1577.1网络优化...............................1577.1.1网络结构多样性.......................1587.1.2高维变量的非凸优化....................1587.1.3神经网络优化的改善方法..................1607.2优化算法...............................1607.2.1小批量梯度下降.......................1607.2.2批量大小选择........................1617.2.3学习率调整..........................1627.2.4梯度估计修正........................1677.2.5优化算法小结........................1707.3参数初始化..............................1717.3.1基于固定方差的参数初始化.................1727.3.2基于方差缩放的参数初始化.................1737.3.3正交初始化..........................1757.4数据预处理..............................1767.5逐层归一化..............................1787.5.1批量归一化..........................1797.5.2层归一化...........................1817.5.3权重归一化..........................1827.5.4局部响应归一化.......................1827.6超参数优化..............................1837.6.1网格搜索...........................1837.6.2随机搜索...........................1847.6.3贝叶斯优化..........................1847.6.4动态资源分配........................1857.6.5神经架构搜索........................1867.7网络正则化..............................1867.7.1?1和?2正则化........................1877.7.2权重衰减...........................1887.7.3提前停止...........................1887.7.4丢弃法............................1897.7.5数据增强...........................1917.7.6标签平滑...........................1917.8总结和深入阅读...........................192第8章 注意力机制与外部记忆1978.1认知神经学中的注意力.......................1988.2注意力机制..............................1998.2.1注意力机制的变体......................2018.3自注意力模型.............................2038.4人脑中的记忆.............................2058.5记忆增强神经网络..........................2078.5.1端到端记忆网络.......................2088.5.2神经图灵机..........................2108.6基于神经动力学的联想记忆.....................2118.6.1Hopfiel网络........................2128.6.2使用联想记忆增加网络容量.................2158.7总结和深入阅读...........................215第9章 无监督学习2199.1无监督特征学习...........................2209.1.1主成分分析..........................2209.1.2稀疏编码...........................2229.1.3自编码器...........................2249.1.4稀疏自编码器........................2259.1.5堆叠自编码器........................2269.1.6降噪自编码器........................2269.2概率密度估计.............................2279.2.1参数密度估计........................2279.2.2非参数密度估计.......................2299.3总结和深入阅读...........................232第10章 模型独立的学习方式23510.1集成学习...............................23510.1.1AdaBoost算法........................23710.2自训练和协同训练..........................24010.2.1自训练............................24010.2.2协同训练...........................24010.3多任务学习..............................24210.4迁移学习...............................24510.4.1归纳迁移学习........................24610.4.2转导迁移学习........................24710.5终身学习...............................24910.6元学习.................................25210.6.1基于优化器的元学习....................25310.6.2模型无关的元学习......................25410.7总结和深入阅读...........................255第三部分 进阶模型第11章 概率图模型26111.1模型表示...............................26211.1.1有向图模型..........................26311.1.2常见的有向图模型......................26411.1.3无向图模型..........................26711.1.4无向图模型的概率分解...................26711.1.5常见的无向图模型......................26911.1.6有向图和无向图之间的转换.................27011.2学习..................................27111.2.1不含隐变量的参数估计...................27111.2.2含隐变量的参数估计....................27311.3推断..................................27911.3.1准确推断...........................27911.3.2近似推断...........................28211.4变分推断...............................28311.5基于采样法的近似推断.......................28511.5.1采样法............................28511.5.2拒绝采样...........................28711.5.3重要性采样..........................28811.5.4马尔可夫链蒙特卡罗方法..................28911.6总结和深入阅读...........................292第12章 深度信念网络29712.1玻尔兹曼机..............................29712.1.1生成模型...........................29912.1.2能量最小化与模拟退火...................30112.1.3参数学习...........................30212.2受限玻尔兹曼机...........................30412.2.1生成模型...........................30512.2.2参数学习...........................30712.2.3受限玻尔兹曼机的类型...................30812.3深度信念网络.............................30912.3.1生成模型...........................31012.3.2参数学习...........................31012.4总结和深入阅读...........................313第13章 深度生成模型31713.1概率生成模型.............................31813.1.1密度估计...........................31813.1.2生成样本...........................31913.1.3应用于监督学习.......................31913.2变分自编码器.............................31913.2.1含隐变量的生成模型....................31913.2.2推断网络...........................32113.2.3生成网络...........................32313.2.4模型汇总...........................32313.2.5再参数化...........................32513.2.6训练..............................32513.3生成对抗网络.............................32713.3.1显式密度模型和隐式密度模型...............32713.3.2网络分解...........................32713.3.3训练..............................32913.3.4一个生成对抗网络的具体实现：DCGAN..........33013.3.5模型分析...........................33013.3.6改进模型...........................33313.4总结和深入阅读...........................336第14章 深度强化学习33914.1强化学习问题.............................34014.1.1典型例子...........................34014.1.2强化学习定义........................34014.1.3马尔可夫决策过程......................34114.1.4强化学习的目标函数....................34314.1.5值函数............................34414.1.6深度强化学习........................34514.2基于值函数的学习方法.......................34614.2.1动态规划算法........................34614.2.2蒙特卡罗方法........................34914.2.3时序差分学习方法......................35014.2.4深度Q网络..........................35314.3基于策略函数的学习方法......................35414.3.1REINFORCE算法......................35614.3.2带基准线的REINFORCE算法...............35614.4演员-评论员算法...........................35814.5总结和深入阅读...........................360第15章 序列生成模型36515.1序列概率模型.............................36615.1.1序列生成...........................36715.2N元统计模型.............................36815.3深度序列模型.............................37015.3.1模型结构...........................37015.3.2参数学习...........................37315.4评价方法...............................37315.4.1困惑度............................37315.4.2BLEU算法..........................37415.4.3ROUGE算法.........................37515.5序列生成模型中的学习问题.....................37515.5.1曝光偏差问题........................37615.5.2训练目标不一致问题....................37715.5.3计算效率问题........................37715.6序列到序列模型...........................38515.6.1基于循环神经网络的序列到序列模型...........38615.6.2基于注意力的序列到序列模型...............38715.6.3基于自注意力的序列到序列模型..............38815.7总结和深入阅读...........................390附录数学基础 393附录A 线性代数 394附录B 微积分 404附录C 数学优化 413附录D 概率论 420附录E 信息论 433索引 439