A Programmer
2023-11-19T04:03:15.124Z
http://www.liaoaoyang.com/
liaoaoyang
Hexo
家庭网络的若干升级
http://www.liaoaoyang.com/articles/2023/11/16/upgrade-home-net/
2023-11-16T13:38:12.000Z
2023-11-19T04:03:15.124Z
<p>近期升级了NAS和家庭的无线网,原本的目的是解决NAS噪音问题,最后的走向却有些偏差。<br>需求的不断变更往往是项目失败的根源,还好在家庭网络升级过程中及时打住。</p>
<a id="more"></a>
<h1 id="起因"><a href="#起因" class="headerlink" title="起因"></a>起因</h1><p>整个事情的起因竟然是为了解决噪音问题。</p>
<p>目前仍在使用古老的QNAP TS-551 NAS,与几年前不同的是,3 HDD 和 2 SSD 都已经插满了。在HDD的选择上,使用了便宜又大碗的 WD HC550 16T,性价比自然是不错,存储容量焦虑大大缓解。SSD 得益于国产固态近半年的降价,可以直接把 SSD 作为存储池的的一部分,但是想要全闪 NAS 的话,目前看起来成本还是太高。</p>
<p>HC550 有一个让人头疼的噪音问题。起停阶段自然不必说,大量读写的时候噪音和震动之大多少有些夸张,然而日常状态会定期出现的“咯噔”声音才是最让人烦躁的存在。这一噪音据说是磁头归位保护硬盘这个特性引起的,但是个人对这个噪音多少有些敏感,虽然实际上不大,深夜情况下,还是有明显的感觉。</p>
<p>考虑了各种办法,最终决定让 NAS 远离自己解决噪音问题。</p>
<h1 id="套壳静音"><a href="#套壳静音" class="headerlink" title="套壳静音"></a>套壳静音</h1><p>首先是套壳阶段。这一阶段主要是尝试使用加装了隔音材料的 MATX 机箱套住 NAS 尝试解决问题。至于说为什么不选择隔音机柜,那是因为成品实在是太贵,大约4位数,这是难以接受的。</p>
<p>为了解决散热问题,用了一把利民的 12 cm PWM 调温风扇配合温控电路板解决,整体成本30元左右。效果上聊胜于无,隐约还是可以听到,主要是不完全封闭,而且 UPS 还在机箱外,整个感觉就是有些不伦不类,这一阶段成果很快就要被否决了。</p>
<h1 id="增加距离"><a href="#增加距离" class="headerlink" title="增加距离"></a>增加距离</h1><p>既然隔音不能完全解决,那么是不是可以考虑让NAS远离?在一个不常有人的空间内部署,应该是最优解了。</p>
<p>不过最大问题是,NAS 的网络如何解决?在一个已经居住的环境里,网线走明线似乎是一个好选择,而且也有扁平网线,看起来哪怕是过门缝这种情况也能应对。如果选择这个方案,30元解决问题。</p>
<p>作为家居环境,还要考虑美观,网线难以做到无感知,地线影响打扫,天花板固定有些突兀,走门缝不一定方便,但是如果结合圆形细线和扁线,用连接器连接,扁线过门缝,圆细线走线,做好部分拐角保护,似乎是一个好的方案。最美观的方案可能就是上3频路由器无线 mesh 了。不过问题是成本,速度,稳定性这三者都需要考虑。如果再给我一次家里新增布线的机会,大概会选择细网线走墙+扁网线过门的方案,过墙部分直接用热熔胶粘平,只要门缝不是太极限(3mm以下)完全没有问题。细网线3mm不到,走墙和直角看起来也不是特别违和,甚至能直接沿着踢脚线布线,二者通过一个法兰连接,看起来问题也不大,这样布线两居室就算全拉上线也才100出头,RJ45的接口也方便接设备。家里有猫狗或者孩子也不用担心,毕竟网线还是很结实,唯一的缺点可能还是能目视到,不过相比起低成本来说,这算得了什么呢?假设家里装修风格是纯白或者边缘是黑色的,可能都不会太意识到。</p>
<p>但是这时候想要折腾的心态突然上来了,觉得这个方案还是不够优雅,而且门缝不一定有超过3mm的高度(扁线还是有厚度的,可能会阻碍关门),目光投向了光纤。</p>
<p>常见的皮线带钢丝,硬度还是有些大,不好塑形,而且还是比较明显。这时候隐形光纤就成为了最佳选择。实际上隐形光纤是接近透明的光纤,去掉了跳线的外壳,大致在1mm左右的线缆,成本上不贵,成品 SC 接口 15 米大约也就是30元,考虑到自己冷接的手艺,买成品即可。隐形光纤最大的问题是不能折90度,在这些地方需要稍微做一些弧度即可。固定可以用热熔胶点胶,等待3s左右防止烫坏光纤,过门缝等地方可以用布基胶布一类的做点简单保护。有条件可以打个光看看有没有漏光的情况,大致判断一下是否有衰减,或者更好的做法是上仪器。</p>
<p>光信号的电信号转换,可以采用光纤收发器解决。TP-LINK 有一对 SC 接口的 FC311A-3 和 FC314B-3 的组合,千兆速度,接收端有4个 RJ45 网口,可以当半个交换机使用,大约 180 左右,是一个不错的选择。</p>
<p>至此,网络问题基本解决了。</p>
<h1 id="机柜组织设备"><a href="#机柜组织设备" class="headerlink" title="机柜组织设备"></a>机柜组织设备</h1><p>NAS 移动到其他空间之后,此时 NAS 的供电和网络等附属设备比较多,看起来比较杂乱,也不利于保护设备,可以考虑上一个机柜。</p>
<p>最终考虑尺寸,选择了 6U 机柜,大概可以放下一台 NAS,一台 UPS,光纤收发器,还剩余大约一个迷你主机的空间。</p>
<p>机柜的供电,散热和走线需要考虑,常规的机柜都是顶部开口,敲掉钢板,然后引入线路。但是考虑到实际上只需要一根电线和光纤,完全不需要这么大的开口,可以考虑机柜的其他较小的空洞引线。散热可以复用阶段1中机箱的PWM风扇,机柜顶部还是有12CM风扇固定位的。供电上可以用自己接线的 PDU 插排,8口总开的,保证有足够的电源入口,不过 PDU 插排会占据1U的高度,这点需要考虑。不过无论使用普通插排还是 PDU 插排自己接线大概率是肯定要的,毕竟三相插头不敲掉顶部钢板很难从机柜中引出,而电线则容易很多。走线的话这个没有特别的说法,不过束线带或者电工胶布还是需要,毕竟混乱的机柜自己用起来也很麻烦。</p>
<p>6U 机柜一般不带轮,用可调节尺寸的台式机支架DIY了一个机柜底座,带可锁定的万向轮。考虑到我是把机柜放在了厨房,垫高机柜还能起到基本的防水目的。</p>
<p>配合米家,增加温度计和智能开关,漏水和高温时自动断电,虽然不能自动灭火,但是还算是稍有改善,也能监测用电量。</p>
<h1 id="扩展应用"><a href="#扩展应用" class="headerlink" title="扩展应用"></a>扩展应用</h1><p>事情到了这个阶段,似乎问题得到了完美解决,但是不要低估个人对需求的创造能力。</p>
<p>既然整个屋子的南北都有了网线,还可以做一些其他什么应用呢?作为一个大进深的屋子,在远离路由器的一端网速堪忧,那么是不是趁机解决一下网络问题?</p>
<p>使用的华硕 AC68U 已经多年,其实至今稳定而且基本够用,更改模式以后,对信号较差的情况有了部分改善。不过唯二的问题是不是很好选择 mesh 方案了,而且无支持 Wi-Fi 6。</p>
<p>还有一个问题是,米家默认情况都是云端执行模式,一旦偶发断网,整个控制都有可能瘫痪。比如 zigbee 开关控制的灯具,这种情况需要一个中枢网关。</p>
<p>综上,首先考虑需要扩展的自然是路由器 mesh 以及米家本地化执行的需求。</p>
<p>刚好近期小米路由器推出了新品6500Pro就是一次性解决这些需求的设备。但是尺寸上属实让人震惊,堪比PS4的尺寸。<br><img src="https://blog.wislay.com/wp-content/uploads/2023/11/blog-6500pro-size.jpeg" alt=""><br>外观算是好看的,没有天线的设计还不错。4个2.5G网口共享5G,目前家用环境影响不大,毕竟内网也就千兆。<br><img src="https://blog.wislay.com/wp-content/uploads/2023/11/blog-6500pro-appearance.jpeg" alt=""><br>能NFC一碰连接,不过仅限于Android。作为路由器的基本的上网功能基本不用担心,Mac Mini千兆无线内网轻松跑满。带了中枢网关和蓝牙网关,搭配多模网关zigbee设备也能用,AP mesh情况下不影响网关使用(上游还有一个主路由器的情况),基本上解决了核心的问题,不过如果带了红外模块就更完美了,目前客厅的其他设备还需要带红外模块音箱进行控制。</p>
<p>正好买到了很便宜的小米 AX3000T,本次方案就是小米路由器主打了。</p>
<p>稳定性上使用了半个月,基本上没有问题,访问 NAS 基本能跑满千兆了。</p>
<p>整个事情到这里已经偏离了最开始的计划。</p>
<h1 id="再提速"><a href="#再提速" class="headerlink" title="再提速"></a>再提速</h1><p>路由器上的2.5G网口只能有千兆速度想起来让人总觉得有些难以接受,最终又想要再次提升速度了。</p>
<p>速度限制的瓶颈其实主要分为三部分,收发两端以及线路。线路上是光纤,自然万兆毫无压力,实际上用6类线在家用距离上也没有问题。但是 USB 5G 网卡的价格确实有些昂贵,更不用说万兆电口网卡了,这类设备也很少。本身Mac上也没有选配万兆网口,倒是也不用这么极限。</p>
<p>采用Realtek 8156芯片的 USB 2.5G 网卡物美价廉,百元左右,很多设备都能驱动,哪怕在老旧的 QNAP 设备上都能免驱。</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2023/11/blog-qnap-2500m-usb-net.jpeg" alt=""></p>
<p>收发段就它了吧。不过问题是光纤收发器是千兆的,这就无法发挥2.5G网卡的能力了。只能升级光纤的收发设备。然而,2.5G的光纤收发器价格也比较贵,不如考虑带光口(SFP)的交换机+光模块的模式了。</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2023/11/blog-xike-sks3200m-4gpy2xf-switch.jpeg" alt=""></p>
<p>兮克的这款 SKS3200M-4GPY2XF 价格尚可,其他大品牌的同类产品稍微有些贵。4电(2.5G)+ 2光(10G)轻管理交换机能支持VLAN,对于家庭组建2.5G内网来说是非常方便的,一条隐形光纤,通过LC SFP模块能完成单线复用等功能。速度应该是不用担心,不过手里没有能跑到10G的设备,2.5G倒是能跑满,配合6500Pro路由器和QNAP NAS上的USB 3.0 2.5G网卡,总算是有了一个准2.5G内网。买上两只组合一下。</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2023/11/blog-10gbps-sfp.jpeg" alt=""></p>
<p>光模块买了一对便宜的单模单纤的万兆,使用LC接口,不过前期的隐形光纤是SC接口,这里需要做一些转接,用法兰连接成品转接线即可。SC接口也看到了有一家有卖单模单纤,但是没有试过。如果当时是两根光纤,购买这类产品的成本会大幅度降低,单模单纤的还是太贵,哪怕是二手设备。</p>
<p>划分了两个 VLAN,其中一个是包含 NAS 的家庭内网。划分VLAN单线复用有个人用一个简单的方式:</p>
<ul>
<li>公用的端口设定为tagged</li>
<li>VLAN本身的端口设定为untagged</li>
<li>VID只设定VLAN本身的端口</li>
</ul>
<p><img src="https://blog.wislay.com/wp-content/uploads/2023/11/blog-xike-strange-web-console.jpeg" alt=""></p>
<p>不过这一交换机有个离谱的交互,交换机居然修改后要到最底部的二级菜单点击保存才可以,这个交互设计过于阴间了,导致第一天配置完成之后mesh子路由器上的IP居然是另一个路由器DHCP分配的,整个家里设备的连接情况一片混乱。接上网线一看VLAN设定全部没了</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2023/11/blog-xike-6500pro-mac-web-speed-test.jpeg" alt=""></p>
<p>看着测速结果,在想原本是要做什么来着?还是立刻打住继续发散的想法吧。</p>
<p>近期升级了NAS和家庭的无线网,原本的目的是解决NAS噪音问题,最后的走向却有些偏差。<br>需求的不断变更往往是项目失败的根源,还好在家庭网络升级过程中及时打住。</p>
IDEA插件开发尝试
http://www.liaoaoyang.com/articles/2021/10/23/idea-plugin-develop-node-1/
2021-10-23T05:57:57.000Z
2021-10-24T02:09:21.635Z
<h1 id="TL-DR"><a href="#TL-DR" class="headerlink" title="TL;DR"></a>TL;DR</h1><p>JetBrains IDEA 基本上是目前最好的 Java 开发 IDE ,根据自己的一些个性化需求定制插件也是可行的。本篇会通过修改一个开源插件的的方式,描述一些简单修改插件的方法以及可能遇到的问题。</p>
<p>此处感谢 <a href="https://github.com/xxxtai/ArthasHotSwap/" target="_blank" rel="external">ArthasHotSwap</a> 插件,这一个插件为开发工作提供了便利。</p>
<a id="more"></a>
<h1 id="场景"><a href="#场景" class="headerlink" title="场景"></a>场景</h1><p>此处以一个场景作为案例。</p>
<p><a href="https://github.com/xxxtai/ArthasHotSwap/" target="_blank" rel="external">ArthasHotSwap</a> 插件是一个使用 Aliyun OSS 作为加密后 class 文件作为中转,近似“一键”完成热更新操作的 IDEA 插件。</p>
<p>插件功能很简单,仅在右侧菜单提供一个 <code>Swap this class</code> 选项:</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2021/10/WX20211023-141000.png" alt="Swap this class"></p>
<p>查看插件源代码:</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// com.<span class="label">xxxtai.arthas.facade.impl.OssFacadeImpl#uploadString</span></span></span><br><span class="line">OSS ossClient = <span class="keyword">new</span> OSSClientBuilder().build(ossInfo.endpoint, ossInfo.accessKeyId, ossInfo.accessKeySecret);</span><br><span class="line">PutObjectRequest putObjectRequest = <span class="keyword">new</span> PutObjectRequest(ossInfo.bucketName, DIRECTORY + key,</span><br><span class="line"> <span class="keyword">new</span> ByteArrayInputStream(content.getBytes()));</span><br><span class="line"></span><br><span class="line">ObjectMetadata metadata = <span class="keyword">new</span> ObjectMetadata();</span><br><span class="line">metadata.setHeader(OSSHeaders.OSS_STORAGE_CLASS, StorageClass.Standard.toString());</span><br><span class="line">metadata.setObjectAcl(CannedAccessControlList.PublicRead);</span><br><span class="line">putObjectRequest.setMetadata(metadata);</span><br><span class="line"></span><br><span class="line">ossClient.putObject(putObjectRequest);</span><br><span class="line">ossClient.shutdown();</span><br></pre></td></tr></table></figure>
<p>可以看到实际上是生成了一个 OSS 上的公共读对象,通过复杂的文件名来提供一定的安全性,并且满足可以在服务器上访问的的链接。</p>
<p>那么下面修改的目标就是<code>提供一个选项,决定是否生成带有过期时间的链接</code>。</p>
<h1 id="开发"><a href="#开发" class="headerlink" title="开发"></a>开发</h1><h2 id="开发环境搭建"><a href="#开发环境搭建" class="headerlink" title="开发环境搭建"></a>开发环境搭建</h2><p>需求明确后先要搭建开发环境。</p>
<p><a href="https://github.com/xxxtai/ArthasHotSwap/" target="_blank" rel="external">ArthasHotSwap</a> 插件使用 gradle 进行工程的组织。</p>
<p>使用 gradle 可能会遇到依赖下载速度缓慢的问题,考虑将 <code>build.gradle</code> 文件中调整一下源:</p>
<figure class="highlight dust"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="xml">repositories </span><span class="expression">{</span><br><span class="line"> /<span class="end-block">/ mavenCentral</span>()</span><br><span class="line"> <span class="variable">maven</span>{ <span class="variable">url</span> '<span class="variable">https</span>:/<span class="end-block">/maven.aliyun.com</span><span class="end-block">/repository</span><span class="end-block">/central</span>/'}</span><span class="xml"></span><br><span class="line"> maven</span><span class="expression">{ <span class="variable">url</span> '<span class="variable">https</span>:/<span class="end-block">/maven.aliyun.com</span><span class="end-block">/repository</span><span class="end-block">/public</span>/'}</span><span class="xml"></span><br><span class="line"> maven</span><span class="expression">{ <span class="variable">url</span> '<span class="variable">https</span>:/<span class="end-block">/maven.aliyun.com</span><span class="end-block">/repository</span><span class="end-block">/google</span>/'}</span><span class="xml"></span><br><span class="line"> maven</span><span class="expression">{ <span class="variable">url</span> '<span class="variable">https</span>:/<span class="end-block">/maven.aliyun.com</span><span class="end-block">/repository</span><span class="end-block">/gradle-plugin</span>/'}</span><span class="xml"></span><br><span class="line">}</span></span><br></pre></td></tr></table></figure>
<p>开发插件还会用到 <a href="https://github.com/JetBrains/gradle-intellij-plugin/" target="_blank" rel="external">gradle-intellij-plugin</a> 。在配置文件中声明的版本可能也需要下载,这个速度也可能很慢,考虑直接替换成本地的 IDEA:</p>
<figure class="highlight dust"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"><span class="xml">// See https://github.com/JetBrains/gradle-intellij-plugin/</span><br><span class="line">intellij </span><span class="expression">{</span><br><span class="line"> /<span class="end-block">/ version </span>'2020<span class="variable">.</span>3<span class="variable">.</span>3'</span><br><span class="line"> /<span class="end-block">/ type </span>'<span class="variable">IC</span>'</span><br><span class="line"> <span class="variable">version</span> '2021<span class="variable">.</span>2'</span><br><span class="line"> <span class="variable">localPath</span> '<span class="end-block">/Users</span><span class="end-block">/l</span><span class="end-block">/Library</span><span class="end-block">/Application Support</span><span class="end-block">/JetBrains</span><span class="end-block">/Toolbox</span><span class="end-block">/apps</span><span class="end-block">/IDEA-U</span><span class="end-block">/ch-</span>0/212<span class="variable">.</span>4746<span class="variable">.</span>92<span class="end-block">/IntelliJ IDEA.app</span>'</span><br><span class="line"> <span class="variable">updateSinceUntilBuild</span> <span class="variable">false</span></span><br><span class="line"> <span class="variable">sameSinceUntilBuild</span> <span class="variable">false</span></span><br><span class="line">}</span><span class="xml"></span></span><br></pre></td></tr></table></figure>
<p>其中 <code>localPath</code> 替换成自己的目录即可,<code>version</code> 也是必须配置的。</p>
<p>这两步完成后可能可以加快投入开发的时间。</p>
<h2 id="插件结构"><a href="#插件结构" class="headerlink" title="插件结构"></a>插件结构</h2><p>开发首先看文档:<a href="https://plugins.jetbrains.com/docs/intellij/plugin-structure.html" target="_blank" rel="external">https://plugins.jetbrains.com/docs/intellij/plugin-structure.html</a></p>
<p>关注 <code>src/main/resources/META-INF/plugin.xml</code>。</p>
<p>一些常规的字段之外,本次修改最关键的部分,就是配置窗口的声明:</p>
<figure class="highlight xml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="tag"><<span class="title">projectConfigurable</span> <span class="attribute">parentId</span>=<span class="value">"tools"</span> <span class="attribute">instance</span>=<span class="value">"com.xxxtai.arthas.dialog.SettingDialog"</span></span><br><span class="line"> <span class="attribute">id</span>=<span class="value">"com.xxxtai.arthas.dialog.SettingDialog"</span> <span class="attribute">displayName</span>=<span class="value">"ArthasHotSwap"</span>/></span></span><br><span class="line"><span class="tag"><<span class="title">projectService</span> <span class="attribute">serviceImplementation</span>=<span class="value">"com.xxxtai.arthas.domain.AppSettingsState"</span>/></span></span><br></pre></td></tr></table></figure>
<p>从这里可以看到入口是 <code>com.xxxtai.arthas.dialog.SettingDialog</code> 这个类。</p>
<p>同时因为目标是提供修改配置的功能,参考文档:<a href="https://plugins.jetbrains.com/docs/intellij/settings-guide.html" target="_blank" rel="external">https://plugins.jetbrains.com/docs/intellij/settings-guide.html</a> 。</p>
<p>结合文档以及配置,<a href="https://github.com/xxxtai/ArthasHotSwap/" target="_blank" rel="external">ArthasHotSwap</a> 插件提供的是项目级别的配置功能,并且目录中位于 Tools 目录下:</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2021/10/WX20211023-145110.png" alt="ArthasHotSwap"></p>
<h2 id="修改方法"><a href="#修改方法" class="headerlink" title="修改方法"></a>修改方法</h2><p>增加一个配置,整体方法就明确了:</p>
<ul>
<li>选择合适的显示组件</li>
<li>增加配置字段</li>
<li>功能层面读取字段完成功能分支</li>
</ul>
<h3 id="选择合适的显示组件"><a href="#选择合适的显示组件" class="headerlink" title="选择合适的显示组件"></a>选择合适的显示组件</h3><p>可以考虑 CheckBox 形式,勾选上表示 true。</p>
<p>注意 <code>com.xxxtai.arthas.dialog.SettingDialog</code> 是配置文件中的入口,而此类会调用 <code>com.xxxtai.arthas.dialog.AppSettingsComponent</code> 对象构建面板。</p>
<p>修改 <code>com.xxxtai.arthas.dialog.AppSettingsComponent</code> ,增加对象以及获取以及赋值方法:</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">AppSettingsComponent</span> </span>{</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> JPanel myMainPanel;</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> JBTextField ossEndpointText = <span class="keyword">new</span> JBTextField();</span><br><span class="line"> <span class="comment">// ...</span></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> JBCheckBox privateAccessRadioButton = <span class="keyword">new</span> JBCheckBox();</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">public</span> <span class="title">AppSettingsComponent</span><span class="params">()</span> </span>{</span><br><span class="line"> myMainPanel = FormBuilder.createFormBuilder()</span><br><span class="line"> .addLabeledComponent(<span class="keyword">new</span> JBLabel(<span class="string">"Enter OSS Endpoint: "</span>), ossEndpointText, <span class="number">1</span>, <span class="keyword">false</span>)</span><br><span class="line"> <span class="comment">// ...</span></span><br><span class="line"> .addLabeledComponent(<span class="keyword">new</span> JBLabel(<span class="string">"Private OSS access: "</span>), privateAccessRadioButton, <span class="number">1</span>, <span class="keyword">false</span>)</span><br><span class="line"> .addComponentFillVertically(<span class="keyword">new</span> JPanel(), <span class="number">0</span>)</span><br><span class="line"> .getPanel();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">public</span> JPanel <span class="title">getPanel</span><span class="params">()</span> </span>{</span><br><span class="line"> <span class="keyword">return</span> myMainPanel;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">public</span> JComponent <span class="title">getPreferredFocusedComponent</span><span class="params">()</span> </span>{</span><br><span class="line"> <span class="keyword">return</span> ossEndpointText;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="annotation">@NotNull</span></span><br><span class="line"> <span class="function"><span class="keyword">public</span> String <span class="title">getOssEndpointText</span><span class="params">()</span> </span>{</span><br><span class="line"> <span class="keyword">return</span> ossEndpointText.getText();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">setOssEndpointText</span><span class="params">(@NotNull String newText)</span> </span>{</span><br><span class="line"> ossEndpointText.setText(newText);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// ...</span></span><br><span class="line"> </span><br><span class="line"> <span class="function"><span class="keyword">public</span> <span class="keyword">boolean</span> <span class="title">getPrivateAccess</span><span class="params">()</span> </span>{</span><br><span class="line"> <span class="keyword">return</span> privateAccessRadioButton.isSelected();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">setPrivateAccess</span><span class="params">(<span class="keyword">boolean</span> privateAccess)</span> </span>{</span><br><span class="line"> privateAccessRadioButton.setSelected(privateAccess);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<h3 id="增加配置字段"><a href="#增加配置字段" class="headerlink" title="增加配置字段"></a>增加配置字段</h3><p><code>com.xxxtai.arthas.dialog.AppSettingsComponent</code> 会在 <code>com.xxxtai.arthas.dialog.SettingDialog</code> 中通过配置文件中的 <code>com.xxxtai.arthas.domain.AppSettingsState</code> 读取和存储配置文件。</p>
<figure class="highlight aspectj"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br></pre></td><td class="code"><pre><span class="line"><span class="annotation">@State</span>(</span><br><span class="line"> name = <span class="string">"com.xxxtai.arthas.domain.AppSettingsState"</span>,</span><br><span class="line"> storages = {<span class="annotation">@Storage</span>(<span class="string">"setting.xml"</span>)}</span><br><span class="line">)</span><br><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">AppSettingsState</span> <span class="keyword">implements</span> <span class="title">PersistentStateComponent</span><<span class="title">AppSettingsState</span>> </span>{</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> String endpoint = <span class="string">""</span>;</span><br><span class="line"> <span class="keyword">public</span> String accessKeyId = <span class="string">""</span>;</span><br><span class="line"> <span class="keyword">public</span> String accessKeySecret = <span class="string">""</span>;</span><br><span class="line"> <span class="keyword">public</span> String bucketName = <span class="string">""</span>;</span><br><span class="line"> <span class="keyword">public</span> String selectJavaProcessName = <span class="string">""</span>;</span><br><span class="line"> <span class="keyword">public</span> String specifyJavaHome = <span class="string">""</span>;</span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">boolean</span> privateAccess = <span class="keyword">false</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="function">AppSettingsState <span class="title">getInstance</span><span class="params">(@NotNull Project project)</span> </span>{</span><br><span class="line"> <span class="function"><span class="keyword">return</span> ServiceManager.<span class="title">getService</span><span class="params">(project, AppSettingsState.class)</span></span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="annotation">@Nullable</span></span><br><span class="line"> <span class="annotation">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="function">AppSettingsState <span class="title">getState</span><span class="params">()</span> </span>{</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">this</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="annotation">@Override</span></span><br><span class="line"> <span class="keyword">public</span> <span class="function"><span class="keyword">void</span> <span class="title">loadState</span><span class="params">(@NotNull AppSettingsState state)</span> </span>{</span><br><span class="line"> XmlSerializerUtil.copyBean(state, <span class="keyword">this</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<p>增加变量之余,上面的 <code>@State</code> 注解也声明了存储位置。</p>
<h3 id="功能层面读取字段完成功能分支"><a href="#功能层面读取字段完成功能分支" class="headerlink" title="功能层面读取字段完成功能分支"></a>功能层面读取字段完成功能分支</h3><p>此处不多描述,即 <code>com.xxxtai.arthas.facade.impl.OssFacadeImpl#uploadString</code> 会给脚本返回 OSS 对象链接,需要加密也是需要在此处理。即读取配置后,决定是否返回加密链接。</p>
<h1 id="测试"><a href="#测试" class="headerlink" title="测试"></a>测试</h1><p>逻辑开发完成之后,项目 gradle 任务中有一个 <code>runIde</code> 任务,运行此任务之后,会启动一个测试 IDEA 实例,可以在此 IDE 中进行功能的测试。</p>
<p>此处可见刚才的私有访问配置已经添加成功:</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2021/10/WX20211023-153849.png" alt="Added checkbox"></p>
<p>配置也存储到<code>@State</code> 注解配置对应文件中:</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2021/10/WX20211023-154020.png" alt="config saved"></p>
<p>至此,一个插件的修改过程结束。安装上可以通过 gradle 打包 fatJar 再进入 IDEA 进行安装即可。</p>
<p>至于如何增加全局配置等其他功能的开发,之后有时间再记录。</p>
<h1 id="参考链接"><a href="#参考链接" class="headerlink" title="参考链接"></a>参考链接</h1><ul>
<li><a href="https://github.com/xxxtai/ArthasHotSwap/" target="_blank" rel="external">ArthasHotSwap</a></li>
<li><a href="https://github.com/JetBrains/gradle-intellij-plugin/" target="_blank" rel="external">gradle-intellij-plugin</a></li>
<li><a href="https://plugins.jetbrains.com/docs/intellij/plugin-structure.html" target="_blank" rel="external">https://plugins.jetbrains.com/docs/intellij/plugin-structure.html</a></li>
<li><a href="https://plugins.jetbrains.com/docs/intellij/settings-guide.html" target="_blank" rel="external">https://plugins.jetbrains.com/docs/intellij/settings-guide.html</a></li>
</ul>
<h1 id="TL-DR"><a href="#TL-DR" class="headerlink" title="TL;DR"></a>TL;DR</h1><p>JetBrains IDEA 基本上是目前最好的 Java 开发 IDE ,根据自己的一些个性化需求定制插件也是可行的。本篇会通过修改一个开源插件的的方式,描述一些简单修改插件的方法以及可能遇到的问题。</p>
<p>此处感谢 <a href="https://github.com/xxxtai/ArthasHotSwap/">ArthasHotSwap</a> 插件,这一个插件为开发工作提供了便利。</p>
白州蒸馏所
http://www.liaoaoyang.com/articles/2019/09/21/hakushu-distillery/
2019-09-21T05:05:26.000Z
2019-09-21T07:14:12.037Z
<h1 id="TL-DR"><a href="#TL-DR" class="headerlink" title="TL;DR"></a>TL;DR</h1><p>如果此次出行只能去一个地方,那当然是白州蒸馏所。</p>
<!-- hakushu-distillery -->
<a id="more"></a>
<h1 id="白州蒸馏所"><a href="#白州蒸馏所" class="headerlink" title="白州蒸馏所"></a>白州蒸馏所</h1><p>两年前参观过<a href="https://liaoaoyang.cn/articles/2017/05/24/a-tour-to-yamazaki-whisky-museum/" target="_blank" rel="external">山崎蒸馏所</a>之后,是时候再打卡下一个了。</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/09/into-the-workshop.jpg" alt=""></p>
<p>三得利旗下除了山崎,还有白州和知多等等品牌,喜欢的波摩还有拉弗格同样也是三得利旗下的品牌。如果在东京附近,那么白州蒸馏所是绝对值得一去的地方。</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/09/suntory-whisky.jpg" alt="suntory-whisky"></p>
<h2 id="地理位置"><a href="#地理位置" class="headerlink" title="地理位置"></a>地理位置</h2><p>白州蒸馏所实际位于东京附近的山梨县小渊泽。从新宿出发,乘坐JR中央本线到达小渊泽站,需要大约2个小时。</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/09/jr-route.jpg" alt="jr-route"></p>
<p>JR每天班次不少,山梨县可能是户外胜地,从新宿来的路上有许多露营、登山爱好者,早上的车是不一定有座位的,可以考虑提前预定指定席,否则很有可能需要站上几乎全程。</p>
<p>再换乘工厂摆渡巴士或者出租车前往,摆渡巴士基本上都是整点附近发车,出JR站后左转看到旅游案内所再往前即可看到站牌,旅游案内所的工作人员可以简单的英语会话,十分热心。当然也可以直接选择附近等候的出租车前往,工厂距离JR站大约6KM。</p>
<p>返程如果错过了一小时一班的摆渡班车或者着急去赶车,可以麻烦工作人员预约出租车,大约需要等待15到20分钟左右,费用约为2700日元。</p>
<p>小渊泽JR站附近店铺开业时间较晚,街上几乎空无一人,并且附近没有便利店,如果在9-10点要吃饭,目前只发现在JR站内二楼右边有一家乌冬面可以选择。</p>
<h2 id="预约"><a href="#预约" class="headerlink" title="预约"></a>预约</h2><p>虽然在现场看到预约的窗口,但是还是可以先在线上预约一下项目。</p>
<p>和山崎蒸馏所类似,白州蒸馏所一样提供<a href="https://webapl.suntory.co.jp/factory/hakushu/?lang=en" target="_blank" rel="external">英文线上预约</a>。¥2000的见学项目与¥1000的见学项目大概是多了更多的品类的试饮。本次时间和其他原因选择了时长大约为80分钟的¥1000见学项目。此处是一个失误,预留给整个游览行程的时间太紧,导致一些其他项目时间不足,比如自主消费的高年份酒品尝,以至于交通上需要打车,建议预留3-4小时可以玩得更加尽兴。</p>
<p>整体项目主要包括背景介绍,原料体验,制作车间与工艺实地探访,熟成酒窖游览,还有最后的试饮环节。可以通过发放的游览材料简单了解。</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/09/hakushu-into.jpg" alt="hakushu-into"></p>
<h2 id="游览"><a href="#游览" class="headerlink" title="游览"></a>游览</h2><p>白州蒸馏所被描述成森林中的工厂名不虚传,高耸的树木在工厂中随处可见,附近环境相当之好,水声隐约可以听见,空气也相当清新。</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/09/stream.jpg" alt="stream"></p>
<p>没有等待太久就到了参观时间,在威士忌博物馆内集合。</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/09/hakushu-museum.jpg" alt="hakushu-museum"></p>
<p>博物馆一面墙壁安装着不同年份的酒桶,虽然是1973年开始,却也营造出了一些历史感。</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/09/bucket-wall.jpg" alt="bucket-wall"></p>
<p>讲解员开始引导大家参观,全程日语,讲解器其实没有太多必要,AppStore搜索“三得利”下载 <strong>工厂参观语音向导</strong> 应用即可。在每一个讲解位置上都有提示牌,按下对应的数字即可。</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/09/app-download.jpg" alt="app-download"></p>
<p>这一次的讲解由平山桑担当,平山桑发现这一组游览者只有我是外国参观者后,几乎每一个讲解点都会提前提醒我切换,可以说非常的友好了。</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/09/mr-hirayama.jpg" alt="mr-hirayama"></p>
<p>从泥煤和大麦开始,朴实无华的原料居然能够造就各个酒厂不同的风味。第一次闻到熏好的大麦,居然有一些肉类的香气!平山桑似乎对南阿尔卑斯山脉硬度30的水很自豪,认为好水造就了白州独特的口感。</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/09/peat.jpg" alt="peat"></p>
<p>下料、发酵之后的就是进入蒸馏器进行蒸馏,也是烈酒产生的必须步骤。</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/09/distillery-workshop.jpg" alt="distillery-workshop"></p>
<p>二次蒸馏就得到威士忌原酒,入桶熟成。熟成区域酒香醉人,这里并不是修辞手法,而是客观感受,因为酒精气息相当之足。也许某一天喝到的白州就从这些酒桶中酿出。</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/09/aging-room.jpg" alt="aging-room"></p>
<p>见学的最后当然也是最为期待的试饮环节之一,高年份酒可以在见学之后在工厂中的Bar自费消费,来得太晚很有可能会提前售罄。在这里仅仅¥2900就可以喝到15ML的25年山崎白州以及30年的響,一定要带上足够的现金,超值!</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/09/try-whisky-n-highball.jpg" alt="try-whisky-n-highball"></p>
<p>有什么能比在白州饮一杯白州更令人愉悦呢?</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/09/hakushu-highball.jpg" alt="hakushu-highball"></p>
<h1 id="TL-DR"><a href="#TL-DR" class="headerlink" title="TL;DR"></a>TL;DR</h1><p>如果此次出行只能去一个地方,那当然是白州蒸馏所。</p>
<!-- hakushu-distillery -->
NAS笔记
http://www.liaoaoyang.com/articles/2019/06/15/a-note-of-nas/
2019-06-15T15:15:05.000Z
2019-09-21T07:14:12.037Z
<h1 id="TL-DR"><a href="#TL-DR" class="headerlink" title="TL;DR"></a>TL;DR</h1><p>近期对NAS充满了兴趣,在这里开始记录家用NAS的一些内容,偏新手和体验向。</p>
<p>近期工作繁忙,有时间就稍微记录一些吧。</p>
<a id="more"></a>
<!-- a-note-of-nas -->
<h1 id="为什么要使用NAS"><a href="#为什么要使用NAS" class="headerlink" title="为什么要使用NAS"></a>为什么要使用NAS</h1><p>早年上学时候设备不多,常规的数据备份迁移需求可以通过移动硬盘解决,也有PT站资源,手机等设备观影体验也不好。</p>
<p>工作后数据逐渐增多,主要是照片以及一些系统备份,硬盘购买数量主键增多,管理相当不便,加上挂过WD绿盘等损失一部分数据之后,养成了定期备份的习惯,手工操作仍然相当麻烦。</p>
<p>个人娱乐方面,早期尝试过小米盒子挂载硬盘以及远程、闲置笔记本等多种方案,最终逐渐变得体验不佳,手机观影也体验不好,逐渐放弃。</p>
<p>综合起来需求大方向就是:</p>
<ul>
<li>备份</li>
<li>多设备数据同步</li>
<li>下载</li>
</ul>
<p>有人可能会说,就上面这些需求,值得入手一个专门的设备去完成这样的工作?闲置的笔电、Mac Mini、其他ITX准系统不能解决吗?</p>
<p>笔电看起来是一个好选择,带有屏幕和输入设备,配合mSATA以及USB 3.0基本上能打满大多数无线网络了,X86架构上几乎能满足任何的需求,看起来只需要一个硬盘柜和一些线材,功耗什么的也都可以接受。</p>
<p>但是个人的一直以来的观点是专物专用,成品NAS的优势是在于不折腾以及有一定的售后和安全相关支持,人生苦短,适度折腾。</p>
<h1 id="选择"><a href="#选择" class="headerlink" title="选择"></a>选择</h1><p>如果按照适度折腾这个前提,各种黑 DSM、黑 QTS 以及退役矿机自组装ITX主机等等都被排除了。</p>
<p>软件上,当然 DSM 广受好评,QTS 系统也可以接受,其他系统就不考虑了。</p>
<p>成品机型,在多天关注后,基本上锁定了群晖或者 QNAP。</p>
<p>机型主要分为架构以及盘位两个维度,ARM 功耗低但是考虑到软件兼容性和可能性,决定放弃,专注X86平台。</p>
<p>盘位从成本上来说,目前机械硬盘价格也不算低,10T酷狼2K+的价格买两个就已经破4K,当然也可以选择WD等移动硬盘拆盘使用,盘位多固然是好事,但是究竟是否有没有这么多数据需要存储?人生需要记忆,但是也需要新的体验,留下可能最精彩的瞬间。</p>
<p>分析了一下2-3年内双盘位基本够用,价格上这类产品也比较亲民,低端机器大多是Celeron处理器,问题不大。</p>
<p>机缘巧合,最终选择了<a href="https://www.qnap.com/zh-cn/product/ts-551" target="_blank" rel="external">QNAP TS-551</a>。</p>
<p>核心硬件参数:</p>
<ul>
<li>3 <em> 3.5 inch + 2 </em> 2.5 inch 盘位设计</li>
<li>Intel Celeron J3355 双核 2.0 GHz 处理器</li>
<li>标配2G DDR3 笔记本低压RAM</li>
<li>双千兆网卡(无内置无线网卡)</li>
</ul>
<p>说说系统的个人观感,DSM 没有深度用过,但是感觉起来更像面向大众用户的产品,十分的友好,界面也比较美观,群晖整体产品感觉很好。相比之下 QTS 就更偏向工程师一些了。</p>
<p>容器与虚拟机这些二者都有,扩展性上充满想象力,但是要认清现实,2K价位只能买到 Celeron 级别的处理器。</p>
<h1 id="上手"><a href="#上手" class="headerlink" title="上手"></a>上手</h1><p>在一番波折后拿到机器,搭配 QNAP 的其他硬件如下:</p>
<ul>
<li>ASUS AC-68U路由</li>
<li>Seagate Iron Wolf 4T NAS硬盘</li>
<li>Crucial MX500 SSD</li>
<li>Team DDR3L 1600 8G RAM</li>
</ul>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/06/qnap-551-with-ram-n-hdd.jpg" alt="qnap-551-with-ram-n-hdd"></p>
<p>都是已有物件,一切以节省为核心。</p>
<p>内存的扩充是重点,否则2G内存基本和虚拟机、Docker等功能告别了。安装很简单,卸下全部硬盘托盘后即可轻松装入。这台机器并不挑内存,试了一下多年以前大学的拆机内存条依然能够使用。</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/06/qnap-551-ram.jpg" alt="qnap-551-ram"></p>
<p>ASUS AC-68U 作为千兆家用路由器对于这台 NAS 来说已经足够了,配上一根6类线也基本够用,想要让 NAS 无线化?看看<a href="https://www.qnap.com/zh-cn/compatibility/" target="_blank" rel="external">兼容列表</a>吧。</p>
<p>主机上有网卡的 MAC 地址,在路由器上固定分配一个 IP 方便使用即可。</p>
<p>安装系统相当简单,局域网按照引导即可,上手毫无压力,半小时左右一台 QNAP NAS 就可以使用了。</p>
<h1 id="问题与解决方案"><a href="#问题与解决方案" class="headerlink" title="问题与解决方案"></a>问题与解决方案</h1><h2 id="DTS支持"><a href="#DTS支持" class="headerlink" title="DTS支持"></a>DTS支持</h2><p>DSM 和 QTS 貌似没有授权,对于这类影片使用默认的 QVideo 无法播放声音。</p>
<p>放弃使用本机播放念头,QNAP TS-551 本机的 HDMI 接口只能说聊胜于无了。</p>
<p>如果使用盒子或者电脑来播放,问题可以解决,但是目前只有一台1代小米盒子,无线播放卡顿程度让人无法忍受,使用电脑等设备挂载 Samba 使用只要不是过老的网卡观看 4K 也问题不大(高压缩高码率未测试),可以考虑更换盒子(连接5G频段播放)。</p>
<p>iOS 则有比较完美的解决方案,只需¥30 购买 nplayer,整体体验上非常完美,从速度到字幕,再到操作都可以说得上基本让人满意的。不花钱的话VLC也可以,但是体验上不如 nplayer。</p>
<p>最好开启 Samba 3,播放拖动会更顺畅。</p>
<h2 id="迅雷下载"><a href="#迅雷下载" class="headerlink" title="迅雷下载"></a>迅雷下载</h2><p>早期迅雷远程可以支持的设备很多,然而17年之后逐渐停止支持。</p>
<p>xware Docker 方案没有使用成功。</p>
<p>如果有下载宝硬件,可以考虑使用 <a href="https://github.com/keli/docker-xiazaibao-xware" target="_blank" rel="external">https://github.com/keli/docker-xiazaibao-xware</a> 这一工具。</p>
<p>目标转向虚拟机方案,通过虚拟机安装 Windows 系统,结合迅雷U享版,映射 NAS 上的目录为网络驱动器,通过远程桌面管理。</p>
<p>这一方案在 QNAP TS-551 上主要存在性能问题,考虑到不能过于影响宿主机性能,分配单核心处理,如果下载并行个数超过2个,虚拟机卡顿到无法使用。事实上,2个任务都已经让 CPU 负载高达 100% 了,鱼与熊掌不可得兼,能用就好。</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/06/qnap-551-thunder-in-vm.jpg" alt="qnap-551-thunder-in-vm"></p>
<p>桌面管理上,推荐直接远程桌面,使用安装Office自带的远程桌面连接工具,还可以做到共享剪贴板,方便添加任务。VNC Viewer也有尝试使用,没有解决共享剪贴板问题,放弃。</p>
<p>同理百度网盘应该也能用类似方式解决。</p>
<p>近期的实验在配备 5G WIFI 网卡之后,通过限制的PC笔记本作为下载机,只使用 NAS 作为存储器的方式效果似乎更好,减轻了 NAS 的负担,NAS 本体后续应该负担更轻量级的应用。</p>
<h2 id="Time-Machine"><a href="#Time-Machine" class="headerlink" title="Time Machine"></a>Time Machine</h2><p>无线的备份 Mac 也是入手 NAS 的巨大动力之一,个人电脑谁也不知道什么时候会遇上硬盘坏掉这个问题,有一些数据真的相对价值很高。</p>
<p>新购入的机器要初始化用时间机器应该也是个好办法,适度折腾就好。</p>
<p>在 App Center 中安装 Hybrid Backup Sync 后<code>备份服务器</code>中选择 Time Machine,设定密码并开启即可,在 Mac 上当然也要选择磁盘,挂载用户名为 TimeMachine,密码是刚刚设置的密码。</p>
<p>首次备份大约需要一个通宵,完成后就可以在家中悄无声息的完成备份。</p>
<p>未完待续,将会逐渐更新。</p>
<h1 id="TL-DR"><a href="#TL-DR" class="headerlink" title="TL;DR"></a>TL;DR</h1><p>近期对NAS充满了兴趣,在这里开始记录家用NAS的一些内容,偏新手和体验向。</p>
<p>近期工作繁忙,有时间就稍微记录一些吧。</p>
模型展
http://www.liaoaoyang.com/articles/2019/04/28/the-2019-beijing-hobby-show/
2019-04-27T16:03:51.000Z
2019-06-23T14:46:25.182Z
<h1 id="TL-DR"><a href="#TL-DR" class="headerlink" title="TL;DR"></a>TL;DR</h1><p>幻想和现实以另一种方式重新演绎,模型真是迷人。</p>
<p>多图杀猫。</p>
<!-- the-2019-beijing-hobby-show -->
<a id="more"></a>
<h1 id="Photos"><a href="#Photos" class="headerlink" title="Photos"></a>Photos</h1><p>随意挑选了一些作品:</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g29bbpshhwj22en1luqvb.jpg" alt=""></p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g29db0z9h0j24002o01l2.jpg" alt=""></p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g29dbktnvkj24002o04qu.jpg" alt=""></p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g29bbnb8gaj22s61kfhdz.jpg" alt=""></p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g29cqw1lzmj230c1p0b2h.jpg" alt=""></p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g29cqx7lqhj22s11kc4qv.jpg" alt=""></p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g29cqyujn3j22p61iqx6v.jpg" alt=""></p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g2bvo92e7lj22qw1ty7wo.jpg" alt=""></p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g2bvo7l8fzj229s1ik1l2.jpg" alt=""></p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g2d0grpvpfj234022r1l8.jpg" alt=""></p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g2alf6j0dvj22u71li4qw.jpg" alt=""></p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g2alf817jpj22t11vdkjs.jpg" alt=""></p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g2alf55tukj22if1oae87.jpg" alt=""></p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g2alfab3swj235s1s1npm.jpg" alt=""></p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g2alfe0wq0j22nd1rlkjs.jpg" alt=""></p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g2alfby21mj22g01mokjq.jpg" alt=""></p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g2alffbi38j21yz1bckjo.jpg" alt=""></p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g2alfhl05qj22k01pce87.jpg" alt=""></p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g2alfj14lkj22wi1xpu15.jpg" alt=""></p>
<h1 id="TL-DR"><a href="#TL-DR" class="headerlink" title="TL;DR"></a>TL;DR</h1><p>幻想和现实以另一种方式重新演绎,模型真是迷人。</p>
<p>多图杀猫。</p>
<!-- the-2019-beijing-hobby-show -->
动物园
http://www.liaoaoyang.com/articles/2019/04/09/visiting-a-zoo/
2019-04-08T16:28:56.000Z
2019-09-21T07:14:12.058Z
<h1 id="TL-DR"><a href="#TL-DR" class="headerlink" title="TL;DR"></a>TL;DR</h1><p>动物还是在自己的栖息地看起来最可爱。</p>
<p>多图杀猫。</p>
<!-- visiting-a-zoo -->
<a id="more"></a>
<h2 id="上海动物园"><a href="#上海动物园" class="headerlink" title="上海动物园"></a>上海动物园</h2><p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g1vagw5djnj230x20phe3.jpg" alt=""></p>
<p>如果在虹桥等车觉得乏了,可以考虑去一下坐地铁只需要20分钟的上海动物园,大约2小时可以基本走上一圈。工作日自然没什么太多亲子出行,在树荫下听着动物园广播中的The Cranberries慢慢走过各个场馆。</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g1vais3piej22hg1nnu12.jpg" alt="红腹角雉"></p>
<p>时隔20年再次走进动物园,设施自然是不新了。但还是被动物的美震撼到了,近距离的感受到了生命和自然的神力。红腹角雉转头过来的一瞬间还是很惊艳。</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g1vakc1082j22x51y47wr.jpg" alt=""></p>
<p>动物园里里不仅仅是人看动物,犀牛也在和做犀牛泥塑的女孩四目相对。</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g1vahmzhknj22jv1p9u14.jpg" alt=""></p>
<p>灵长类动物在动物园里总是感觉有点悲伤。对动物的介绍也只有寥寥百字。</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g1vajav0g1j22if1obkjr.jpg" alt=""></p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g1vai56y5fj22411ep7wn.jpg" alt=""></p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g1wdfa4mryj22f11m1u12.jpg" alt=""></p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g1wdf1mtrbj21ax0va7wi.jpg" alt=""></p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g1wdeeeaflj22ug1wbe8a.jpg" alt=""></p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g1wdeqrx7yj22jr1p6kjr.jpg" alt=""></p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g1wdejpiyej22t51vgkjs.jpg" alt=""></p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g1vafvb079j22hg1nnqva.jpg" alt=""></p>
<h2 id="北京动物园"><a href="#北京动物园" class="headerlink" title="北京动物园"></a>北京动物园</h2><p>在西直门附近上学,多年来却从没去过动物园,临时起意决定去看看。</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g2ahnmr0xtj22e31lf7wm.jpg" alt=""></p>
<p>首都动物园自然历史悠久,设施也能看出时间的痕迹了。中午到售票处已经有近三万人入园了……门票倒是很亲民,旺季连同熊猫馆门票还不够吃个牛肉面加一份肉。</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g2ahn78froj235s23v7wq.jpg" alt=""></p>
<p>早上看到灰鹦鹉的消息,一直好奇到底是什么样的美妙生物,没想到充满质感的灰色羽毛之下还有红色的尾羽!</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g2ahnhd9khj22k91pjb2g.jpg" alt=""></p>
<p>经过紫蓝金刚鹦鹉,沉默的鹦鹉看到一位大叔突然激动,上下翻飞最后一个倒挂和大叔玩了起来。原来大叔是它98年刚引进时的第一任饲养员,来见老朋友了。紫蓝金刚鹦鹉可以活90岁,大叔说它是濒危动物,也一直没伴儿,有一些抑郁了。时间有限,临走时听到大叔说这个月去它原产地巴西,不知道是不是去给它找个伴儿?果然生物都需要有所羁绊……</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g2ahyxyh70j235s23vqvf.jpg" alt=""></p>
<p>鹦鹉应该是最适合拍照的鸟类之一了。</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g2bvcpa5lpj22yf1yznpo.jpg" alt=""></p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g2bvcqus7xj22ip1oh4qw.jpg" alt=""></p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g2bvcsd4rxj22nq1rvqvd.jpg" alt=""></p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g2bvctjiwcj22kz1pz7wn.jpg" alt=""></p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g2bvcv9u55j22q31tfx6w.jpg" alt=""></p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g2bvcmz8twj235s23v4r3.jpg" alt=""></p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g2bvgelpv6j22g91mvnpi.jpg" alt=""></p>
<p>如果说有什么最让人羡慕的,那当然是胖达和河马了,对它们来说,哪一天不是周末呢?</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g2ahnushuoj235s23vhe3.jpg" alt=""></p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g2ahnzouozj22k91pj7wn.jpg" alt=""></p>
<p>然而狐猴都在仰望天空,明天还是打起精神去好好工作。</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2019/05/727517e7ly1g2ahomm5wpj235s23vb2i.jpg" alt=""></p>
<h1 id="END"><a href="#END" class="headerlink" title="END"></a>END</h1><p>动物还是在自己的栖息地看起来最可爱。 </p>
<p>对动物表演还是不忍观看。</p>
<h1 id="TL-DR"><a href="#TL-DR" class="headerlink" title="TL;DR"></a>TL;DR</h1><p>动物还是在自己的栖息地看起来最可爱。</p>
<p>多图杀猫。</p>
<!-- visiting-a-zoo -->
微信多回调域名
http://www.liaoaoyang.com/articles/2019/01/28/mutilple-wechat-oauth-callback-urls/
2019-01-28T15:27:34.000Z
2019-06-23T14:46:25.182Z
<h1 id="TL-DR"><a href="#TL-DR" class="headerlink" title="TL;DR"></a>TL;DR</h1><p>只有一个公众号但需要让多个域名获取OAuth返回的用户信息或微信登录功能,可以考虑使用集中的回调域名进行中转。</p>
<p>本文也是对个人实验项目 <a href="https://github.com/liaoaoyang/swan" target="_blank" rel="external">SWAN</a> 相关功能的说明。</p>
<!-- mutilple-wechat-oauth-callback-urls -->
<a id="more"></a>
<h1 id="分析"><a href="#分析" class="headerlink" title="分析"></a>分析</h1><p>参见微信开发文档<a href="https://mp.weixin.qq.com/wiki?t=resource/res_main&id=mp1421140842" target="_blank" rel="external">微信网页授权</a>,需要在后台配置<code>redirect_uri</code>,但是<code>redirect_uri</code>每月配置次数以及个数都是有限的。</p>
<p>如果业务有多个域名,都需要使用微信登录或者获取微信基础用户信息,就会存在问题。</p>
<p>整个流程中,配置的<code>redirect_uri</code>是掌握在自己手中的,可以考虑使用回调url再做一次重定向,将用户信息带给其他域名。</p>
<h1 id="实现"><a href="#实现" class="headerlink" title="实现"></a>实现</h1><h2 id="时序图"><a href="#时序图" class="headerlink" title="时序图"></a>时序图</h2><p>以个人目中的实现 <a href="https://github.com/liaoaoyang/swan/blob/master/app/Utils/MyWXTAuth.php" target="_blank" rel="external">MyWXTAuth.php</a> 举例:</p>
<figure class="highlight gherkin"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br></pre></td><td class="code"><pre><span class="line">,----. ,---. ,----. ,------.</span><br><span class="line">|<span class="string">User</span>|<span class="string"> </span>|<span class="string">T3P</span>|<span class="string"> </span>|<span class="string">SWAN</span>|<span class="string"> </span>|<span class="string">WeChat</span>|</span><br><span class="line">`-+--' `-+-' `-+--' `--+---'</span><br><span class="line"> |<span class="string"> </span>|<span class="string"> </span>|<span class="string"> </span>|<span class="string"> </span><br><span class="line"> </span>|<span class="string"> -------------></span>|<span class="string"> </span>|<span class="string"> </span>|<span class="string"> </span><br><span class="line"> </span>|<span class="string"> </span>|<span class="string"> </span>|<span class="string"> </span>|<span class="string"> </span><br><span class="line"> </span>|<span class="string"> </span>|<span class="string">bid/url/key/scope</span>|<span class="string"> </span>|<span class="string"> </span><br><span class="line"> </span>|<span class="string"> </span>|<span class="string">-----------------> </span>|<span class="string"> </span><br><span class="line"> </span>|<span class="string"> </span>|<span class="string"> </span>|<span class="string"> </span>|<span class="string"> </span><br><span class="line"> </span>|<span class="string"> </span>|<span class="string"> </span>|<span class="string"> OAuth callback url/code/...</span>|<span class="string"> </span><br><span class="line"> </span>|<span class="string"> </span>|<span class="string"> </span>|<span class="string"> ---------------------------> </span><br><span class="line"> </span>|<span class="string"> </span>|<span class="string"> </span>|<span class="string"> </span>|<span class="string"> </span><br><span class="line"> </span>|<span class="string"> </span>|<span class="string"> </span>|<span class="string"> </span>|<span class="string"> </span><br><span class="line"> </span>|<span class="string"> <------------------------------------------------------------- </span><br><span class="line"> </span>|<span class="string"> </span>|<span class="string"> </span>|<span class="string"> </span>|<span class="string"> </span><br><span class="line"> </span>|<span class="string"> </span>|<span class="string"> confirm </span>|<span class="string"> </span><br><span class="line"> </span>|<span class="string"> -------------------------------------------------------------> </span><br><span class="line"> </span>|<span class="string"> </span>|<span class="string"> </span>|<span class="string"> </span>|<span class="string"> </span><br><span class="line"> </span>|<span class="string"> </span>|<span class="string"> </span>|<span class="string"> user information </span>|<span class="string"> </span><br><span class="line"> </span>|<span class="string"> </span>|<span class="string"> </span>|<span class="string"> <--------------------------- </span><br><span class="line"> </span>|<span class="string"> </span>|<span class="string"> </span>|<span class="string"> </span>|<span class="string"> </span><br><span class="line"> </span>|<span class="string"> </span>|<span class="string">user information </span>|<span class="string"> </span>|<span class="string"> </span><br><span class="line"> </span>|<span class="string"> </span>|<span class="string"><----------------- </span>|<span class="string"> </span><br><span class="line">,-+--. ,-+-. ,-+--. ,--+---.</span><br><span class="line"></span>|<span class="string">User</span>|<span class="string"> </span>|<span class="string">T3P</span>|<span class="string"> </span>|<span class="string">SWAN</span>|<span class="string"> </span>|<span class="string">WeChat</span>|</span><br><span class="line">`----' `---' `----' `------'</span><br></pre></td></tr></table></figure>
<h2 id="步骤"><a href="#步骤" class="headerlink" title="步骤"></a>步骤</h2><h3 id="请求SWAN授权URL"><a href="#请求SWAN授权URL" class="headerlink" title="请求SWAN授权URL"></a>请求SWAN授权URL</h3><p>常规微信登录可以访问拼接好的微信相关URL,此处修改为类似的方式,访问 SWAN 实现的授权URL,参数说明:</p>
<h4 id="bid"><a href="#bid" class="headerlink" title="bid"></a>bid</h4><p>业务ID,SWAN 会通过这一参数,读取配置信息中允许跳转的域名。</p>
<h4 id="url"><a href="#url" class="headerlink" title="url"></a>url</h4><p>业务方的回跳URL。在微信授权成功后,会在回跳URL中追加一个参数<code>wx_tauth_data</code>,参数内容为JSON格式的用户信息</p>
<h4 id="key"><a href="#key" class="headerlink" title="key"></a>key</h4><p>可以为随机数。</p>
<h4 id="scope"><a href="#scope" class="headerlink" title="scope"></a>scope</h4><p>即微信 OAuth 中的 scope 值,即<code>snsapi_base</code> 与 <code>snsapi_userinfo</code>。</p>
<p>当 <code>scope</code> 为 <code>snsapi_base</code> 时,会使用静默登录,返回的数据格式为:</p>
<figure class="highlight json"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">{"<span class="attribute">id</span>":<span class="value"><span class="string">"oatEjwAVy4H_1FsftYTBD2beKwBE"</span></span>}</span><br></pre></td></tr></table></figure>
<p>如果需要比如用户头像等信息,需要将 <code>scope</code> 设定为 <code>snsapi_userinfo</code> 并进入授权模式。</p>
<p>返回数据格式样例为:</p>
<figure class="highlight json"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">{</span><br><span class="line">"<span class="attribute">openid</span>": <span class="value"><span class="string">"oatEjwAVy4H_1FsftYTBD2beKwBE"</span></span>,</span><br><span class="line">"<span class="attribute">nickname</span>": <span class="value"><span class="string">"TEST"</span></span>,</span><br><span class="line">"<span class="attribute">sex</span>": <span class="value"><span class="number">1</span></span>,</span><br><span class="line">"<span class="attribute">language</span>": <span class="value"><span class="string">"en"</span></span>,</span><br><span class="line">"<span class="attribute">city</span>": <span class="value"><span class="string">"朝阳"</span></span>,</span><br><span class="line">"<span class="attribute">province</span>": <span class="value"><span class="string">"北京"</span></span>,</span><br><span class="line">"<span class="attribute">country</span>": <span class="value"><span class="string">"中国"</span></span>,</span><br><span class="line">"<span class="attribute">headimgurl</span>": <span class="value"><span class="string">"http://thirdwx.qlogo.cn/mmopen/vi_32/Q0j4dwGTeTIeRzJ65COm8ia2MMpjbNUo0GtDJWy4g1dJHDClV6BmMt9gr5gDicHXxJsyB1GicSiae2rxkIE5LfsCmA/132"</span></span>,</span><br><span class="line">"<span class="attribute">privilege</span>": <span class="value">[]</span><br><span class="line"></span>}</span><br></pre></td></tr></table></figure>
<h3 id="举例"><a href="#举例" class="headerlink" title="举例"></a>举例</h3><p>假设域名为 <code>wxtauth.foo.com</code>,<code>bid</code> 为 <code>test</code>,回调地址 <code>url</code> 为 <code>https://wxtauth.foo.com/callback</code>,<code>scope</code> 为 <code>snsapi_userinfo</code>,客户端和服务端需要完成的工作为:</p>
<h4 id="客户端"><a href="#客户端" class="headerlink" title="客户端"></a>客户端</h4><p>假设服务端的域名为 <code>swan.bar.com</code>,默认情况下(假设服务端使用HTTPS),用于中转的登录接口 URL 为 <code>https://swan.bar.com/wechat/swan/wxtauth</code>。</p>
<p>完成自己的业务逻辑之后拼接出 URL:</p>
<p> <code>https://swan.bar.com/wechat/swan/wxtauth?bid=test&url=https://wxtauth.foo.com/callback&scope=snsapi_userinfo&key=123456</code></p>
<p> 并302重定向到这一 URL 之上。</p>
<p>在弹出新窗口中完成授权操作后,需要在 <code>url</code> 参数指定的 URL 上实现相应的登录成功处理逻辑,即读取回调中的携带的 <code>wx_tauth_data</code> 参数,之后继续完成对应的业务逻辑。</p>
<h4 id="服务端"><a href="#服务端" class="headerlink" title="服务端"></a>服务端</h4><p>处理逻辑参见 <a href="https://github.com/liaoaoyang/swan/blob/master/app/Http/Controllers/WeChatController.php" target="_blank" rel="external">\App\Http\Controllers\WeChatController::wxtauth</a>。</p>
<p>服务端首先需要在 <code>.env</code> 文件中配置 <code>bid</code> 为 <code>test</code> 的请求所允许的回调域名。配置格式:</p>
<p>WX_TAUTH<em>CLIENT</em><strong><em>TEST</em></strong>_AUTHENTIC_DOMAINS=wxtauth.foo.com</p>
<p>请注意加粗的 <code>TEST</code> ,每新增一个bid,需要增加一行配置,在加粗部分替换为大写的 <code>bid</code> 的值。</p>
<p>之后服务端会完成微信相关的授权功能,这里特别感谢 <a href="https://github.com/overtrue" target="_blank" rel="external">安正超</a> 优秀的开源项目 <a href="https://github.com/overtrue/wechat" target="_blank" rel="external">EasyWeChat</a>,让微信接入变得极为简便。</p>
<h1 id="TL-DR"><a href="#TL-DR" class="headerlink" title="TL;DR"></a>TL;DR</h1><p>只有一个公众号但需要让多个域名获取OAuth返回的用户信息或微信登录功能,可以考虑使用集中的回调域名进行中转。</p>
<p>本文也是对个人实验项目 <a href="https://github.com/liaoaoyang/swan">SWAN</a> 相关功能的说明。</p>
<!-- mutilple-wechat-oauth-callback-urls -->
发号器设计简述
http://www.liaoaoyang.com/articles/2018/12/25/id-generator/
2018-12-25T15:00:00.000Z
2019-05-18T17:59:04.956Z
<h1 id="TL-DR"><a href="#TL-DR" class="headerlink" title="TL;DR"></a>TL;DR</h1><p>发号器用于为系统中各个模块提供ID生成服务。</p>
<p>设计一个发号器至少需要考虑ID唯一性,有序性,性能,可用性,使用成本,以及一定的数据安全问题。</p>
<p>作为一个极其成熟的业务功能,发号器有大量的实现,有海量的文章描述各类要点,按需选择即可,无需重复造轮子。</p>
<!-- id-generator -->
<a id="more"></a>
<h1 id="为什么不使用UUID?"><a href="#为什么不使用UUID?" class="headerlink" title="为什么不使用UUID?"></a>为什么不使用UUID?</h1><p><a href="https://zh.wikipedia.org/wiki/%E9%80%9A%E7%94%A8%E5%94%AF%E4%B8%80%E8%AF%86%E5%88%AB%E7%A0%81" target="_blank" rel="external">UUID</a> 算法上使用部分硬件信息,从前面提到的几个维度来分析一下:</p>
<ul>
<li>唯一性上差强人意(虚假的硬件信息等)</li>
<li>有序性上不完全能够保证</li>
<li>性能上由于可以是本地生成没有太大问题</li>
<li>可用性亦然</li>
<li>使用成本上开发成本低,然而占用空间等较大</li>
<li>安全性上不能被遍历,高版本的生成算法可以让机器无法回溯</li>
</ul>
<p>综上,使用 UUID 带来的问题首先就是有序性上考量让人质疑,同时占用的空间较为巨大,生成的ID也较难理解,使用了不同算法实现的UUID需要深入实现才能人工制造出一个同类型的UUID。</p>
<p>最直接的影响,就是当前大量系统使用数据库,UUID 作为主键会<a href="http://imysql.com/2014/09/14/mysql-faq-why-innodb-table-using-autoinc-int-as-pk.shtml" target="_blank" rel="external">严重影响性能</a>。</p>
<p>并不是说不能使用UUID,而是需要慎重考虑,属于架构上的取舍。</p>
<h1 id="解决问题"><a href="#解决问题" class="headerlink" title="解决问题"></a>解决问题</h1><p>合理设计的发号器可以解决如下问题(部分):</p>
<h2 id="分库分表的分区键"><a href="#分库分表的分区键" class="headerlink" title="分库分表的分区键"></a>分库分表的分区键</h2><p>业务在初始阶段往往是一个单体应用,往往选择依赖数据库生成自增ID来标示新增数据。</p>
<p>随着业务增长,单表数据增长到千万及以上,或者出现了激烈的竞争条件,需要考虑进行分库分表操作。</p>
<p>此时的ID成了问题,通过发号器为每条记录生成一个全局唯一的ID,可以避免不必要的主键冲突等问题,让数据各得其所。</p>
<h2 id="自增ID的数据安全性"><a href="#自增ID的数据安全性" class="headerlink" title="自增ID的数据安全性"></a>自增ID的数据安全性</h2><p>例如订单相关的业务,每日的流水属于业务的机密数据,如果直接选用自增ID作为订单号,竞对很容易直接估算出每天的下单量(例如间隔24小时下单)。</p>
<p>通过特定算法实现的发号器,一定程度可以规避这一问题。</p>
<h2 id="ID的可读性"><a href="#ID的可读性" class="headerlink" title="ID的可读性"></a>ID的可读性</h2><p>业务上的ID如果单纯是数字的信息量就太小了。</p>
<p>可以在ID中包含一些业务信息,帮助快速的定位问题。</p>
<h1 id="常见架构"><a href="#常见架构" class="headerlink" title="常见架构"></a>常见架构</h1><h2 id="生成算法"><a href="#生成算法" class="headerlink" title="生成算法"></a>生成算法</h2><p>大量的发号器设计上都使用 <a href="https://github.com/twitter-archive/snowflake" target="_blank" rel="external">Snowflake</a> 作为ID生成算法。</p>
<p>核心就是<code>时间戳</code>+<code>节点信息</code>+<code>业务数据</code>,通过位运算组合,在10进制模式下,数字基本单调递增而不连续。</p>
<p>生成的 ID 是64位数字,存储空间和表现力上来说对于绝大多数系统足够。</p>
<h3 id="时间戳"><a href="#时间戳" class="headerlink" title="时间戳"></a>时间戳</h3><p>默认的算法是41位用于毫秒级别的时间戳,通常还会搭配一个起始时间(默认是时间戳的0点),可供使用<code>相对</code>的69年时间,日常使用中可以考虑缩减,腾出更多空间用于业务数据,毕竟6年之后的事情都难以预知。</p>
<h3 id="节点信息"><a href="#节点信息" class="headerlink" title="节点信息"></a>节点信息</h3><p>一般每个服务器分配一个编号,可以结合 ZK / Etcd 等分布式协同组件管理编号,机器本身也可以配置一个默认编号用于降级,多台服务器使用一个编号主要带来的问题就是冲突。</p>
<h3 id="业务数据"><a href="#业务数据" class="headerlink" title="业务数据"></a>业务数据</h3><p>业务数据包含两个方面,一个是业务区分信息,即表示这一类型的ID是何种ID,便于查询问题,能够迅速判断应当从哪一系统入手。</p>
<p>另一个方面自然是核心的编号,也就是自增的数字序列。</p>
<p>虽然已经做了<code>时间</code>(ID 中携带时间戳)以及<code>空间</code>(ID 中的节点信息)上的隔离,但是限于 ID 的数据长度,1秒(或者1毫秒)内同一节点上生成的是有限个的,取决于数据长度。</p>
<h2 id="架构设计"><a href="#架构设计" class="headerlink" title="架构设计"></a>架构设计</h2><p>为了实现全局唯一,需要一个中心节点。</p>
<p>从对中心节点的依赖程度出发,可以分为两类实现,</p>
<h3 id="纯中心实现"><a href="#纯中心实现" class="headerlink" title="纯中心实现"></a>纯中心实现</h3><p>对于强依赖类型的发号器,每个 ID 的生成需要从中心节点获取。</p>
<p>最常见的实现方式就是直接使用数据库的自增 ID 生成序列,或者基于 Redis 这类 NoSQL 的内置特性完成。</p>
<p>一般采用这类方案的问题在于存在单点问题以及性能有上限(然而又有多少系统可以达到系统性能的上限呢?),项目中的订单系统实战过自增 ID + Snowflake的方案。</p>
<p>基于 Redis 的自增 ID 方案可以参考 <a href="https://weibo.com/tangfl" target="_blank" rel="external">唐福林</a> 老师的博文《6行代码实现一个 id 发号器》,结合 Lua 实现,言简意赅。由于原文链接已经失效,这里摘抄一下:</p>
<figure class="highlight smali"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br></pre></td><td class="code"><pre><span class="line">id 发号器的问题, @一乐 的这篇文章说的很透彻了: http://weibo.com/p/1001603800404851831206 但参考实现就显得有些复杂。最近在雪球工作中正好需要用到发号器,于是用 Lua 在 Redis 上实现了一个最简单的:</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">-- usage: redis-cli -h 10.10.201.100 -p 10401 EVAL <span class="string">"$(cat getID.lua)"</span> 1 XID:01:02 </span><br><span class="line">local arg = KEYS<span class="keyword">[</span>1] </span><br><span class="line">-- project id must has 2 digits, 01 - 15 </span><br><span class="line">local pid =<span class="function"> tonumber(</span>string.sub(arg, 5, 6<span class="function">)</span><span class="function">)</span> </span><br><span class="line">--<span class="instruction"> instance </span>id must has 2 digits, 01 - 15 </span><br><span class="line">local iid =<span class="function"> tonumber(</span>string.sub(arg, 8, 9<span class="function">)</span><span class="function">)</span> </span><br><span class="line">local idnum =<span class="function"> redis.call(</span><span class="string">"INCR"</span>, <span class="string">"ID_IDX"</span><span class="function">)</span> % 65536 </span><br><span class="line">local sec =<span class="function"> redis.call(</span><span class="string">"TIME"</span><span class="function">)</span><span class="keyword">[</span>1] - 1420041600 <span class="instruction"></span><br><span class="line">return </span>sec*16777216 + pid*1048576 + iid*65536 + idnum </span><br><span class="line"></span><br><span class="line">解释:</span><br><span class="line"></span><br><span class="line">id 总长度 52bit,为了兼容 js,php,flex 等语言<span class="instruction"> long </span>类型最长只能 52 bit</span><br><span class="line">最高 28 bit 为秒级时间戳,因为位数限制,不能从 1970.1.1 开始,-1420041600 表示从 2015.1.1 开始,大约可以使用10年(3106天)</span><br><span class="line">接下来4个bit为 project id,客户端传入,区分业务</span><br><span class="line">再接下来4个bit为<span class="instruction"> instance </span>id,HA 用的,支持最多16个instance。如果业务只需要“秒级粗略有序”,比如发微博或发评论,则可以多个<span class="instruction"> instance </span>同时使用,不需要做任何特殊处理;如果业务需要“因果有序”,比如某个user短期内快速做的多个操作必须有因果顺序(程序化交易,必须先卖再买,几个毫秒内完成),那么就需要做一些特殊处理,要么用 uid 做一致性hash,或者像我们这样偷懒:一段时间内固定使用一个<span class="instruction"> instance</span><br><span class="line"></span>最低16个bit是 sequence id,每个<span class="instruction"> instance </span>支持每秒 65535 个 id。bit数再大,redis 该成为瓶颈了。</span><br><span class="line">twitter和微博都是把<span class="instruction"> instance </span>id 写死到 server 端,这样 server端就变成有状态的了:16个instance,每个<span class="instruction"> instance </span>都与其它的配置不一样。在雪球我们不希望 server 端有状态,于是设计成<span class="instruction"> instance </span>id 由客户端传入,server 端退化成一个普通的 redis cache server (不需要 rdb,不需要 aof,宕机重启即可)</span><br><span class="line">几个注意点:宕机不能自动立即重启,必须间隔1秒以上,避免 sequence id 重复导致 id 重复;迁移时必须先 kill 老的 instance,再启动新的,也是为了避免 sequence id 重复。</span><br><span class="line">id 发号器说简单也确实挺简单。任何一个技术点,只要理解了本质,大约都是这么简单罢。</span><br></pre></td></tr></table></figure>
<p>这类方案几乎能满足中小业务的需求了,成本上可以复用现有基础设施,如果是做成了公共服务,成本会进一步降低。</p>
<p>纯中心实现有一个优势,就是所有ID能保证全局单调递增。</p>
<h3 id="中心-客户端实现"><a href="#中心-客户端实现" class="headerlink" title="中心-客户端实现"></a>中心-客户端实现</h3><p>这部分又可以分为<code>序列中心分配</code>和<code>序列客户端自行生成</code>两类:</p>
<h4 id="序列中心分配"><a href="#序列中心分配" class="headerlink" title="序列中心分配"></a>序列中心分配</h4><p>这类方案可以参考 <a href="https://github.com/Meituan-Dianping/Leaf" target="_blank" rel="external">Leaf</a> 。简单来说,有如下一些特点:</p>
<ul>
<li>通过预先分配 ID 区间解决性能问题,每次按照特定步长分配ID,客户端缓存区间本地生成 ID</li>
<li>通过为不同客户端分配不同区间解决 ID 唯一性问题,在客户端内实现 ID 的单调递增</li>
<li>结合 Snowflake 算法解决 ID 信息安全性问题</li>
</ul>
<p>在某些数据库中间件中也会提供类似的方案,专门的发号系统还会增加如下的一些组件,解决部分问题:</p>
<ul>
<li>使用 ZK/Etcd 保证节点信息的唯一性</li>
<li>引入时钟修正机制解决节点时间错乱问题,保证 ID 全局唯一</li>
<li>预先获取下一区间降低TP999,减少系统响应时间毛刺</li>
<li>同机房优先调用/中心节点自动切换等运维手段提升中心的可用性</li>
</ul>
<p>这类方案仍然需要中心节点实现高可用,但是极大程度的降低了中心节点的压力。</p>
<p>然而,这类方案也存在一些问题:</p>
<ul>
<li>难于在线调整步长</li>
<li>闰秒的处理(几乎所有基于时间的方案都需要注意)</li>
<li>对序列数据直接使用仍然有可能猜测出业务量级</li>
<li>ID 只是局部单调递增</li>
<li>有大量组件依赖,适合做成公共服务</li>
</ul>
<p>由于需要客户端缓存,对于 PHP 这类非常驻进程的语言,可以考虑引入共享内存完成功能。</p>
<h4 id="序列客户端自行生成"><a href="#序列客户端自行生成" class="headerlink" title="序列客户端自行生成"></a>序列客户端自行生成</h4><p>如果固定了序列的位数,同时引入分布式协同组件,为客户端提供时间校正以及分配节点ID,结合 Snowflake 算法,可以在客户端自定生成序列。这类方案的优点在于:</p>
<ul>
<li>去除了数据库等序列生成中心组件的依赖</li>
</ul>
<p>这一类方案的问题是,1个时间片内(1s/1ms)序列使用完毕后,不能再生成新的ID,为了防止覆盖已生成ID,需要等到下一个时间片内才能继续生成。</p>
<p>另一个问题是,单个节点上存在多个应用时,面临着多个应用竞争同一个 ID 的情况,需要通过一些锁定的手段,保证序列正常发放。当然,如果节点的概念是每个应用,则这个问题可以忽略,但是带来的问题就是分配的节点信息的位数可能需要增加。</p>
<p>虽然去除了数据库的依赖,仍然存在 ZK / Etcd 等协同组件的依赖,对于小型业务系统来说,在有限节点的情况下,可以考虑直接使用 IP 转化为节点 ID,结合 NTP 服务器,去除分布式协同组件的依赖,彻底实现分布式的 ID 生成。</p>
<h1 id="其他"><a href="#其他" class="headerlink" title="其他"></a>其他</h1><ul>
<li><a href="http://ericliang.info/2015/01/18/what-kind-of-id-generator-we-need-in-business-systems.html" target="_blank" rel="external">业务系统需要怎样的全局唯一ID?</a></li>
<li><a href="https://tech.meituan.com/2017/04/21/mt-leaf.html" target="_blank" rel="external">Leaf——美团点评分布式ID生成系统</a></li>
<li><a href="https://github.com/baidu/uid-generator/blob/master/README.zh_cn.md" target="_blank" rel="external">baidu/uid-generator</a> </li>
<li><a href="https://www.infoq.cn/article/wechat-serial-number-generator-architecture" target="_blank" rel="external">微信序列号生成器架构设计及演变</a></li>
<li><a href="http://www.10tiao.com/html/773/201712/2247487246/2.html" target="_blank" rel="external">分布式ID生成系统怎么做</a></li>
<li><a href="https://tech.youzan.com/id_generator/" target="_blank" rel="external">如何做一个靠谱的发号器</a></li>
<li><a href="https://github.com/ramsey/uuid" target="_blank" rel="external">PHP ramsey/uuid</a></li>
</ul>
<h1 id="TL-DR"><a href="#TL-DR" class="headerlink" title="TL;DR"></a>TL;DR</h1><p>发号器用于为系统中各个模块提供ID生成服务。</p>
<p>设计一个发号器至少需要考虑ID唯一性,有序性,性能,可用性,使用成本,以及一定的数据安全问题。</p>
<p>作为一个极其成熟的业务功能,发号器有大量的实现,有海量的文章描述各类要点,按需选择即可,无需重复造轮子。</p>
<!-- id-generator -->
抓包那点事儿(常用工具简介)
http://www.liaoaoyang.com/articles/2018/11/01/packet-capture-i/
2018-11-01T12:00:00.000Z
2019-01-18T16:47:16.983Z
<h1 id="TL-DR"><a href="#TL-DR" class="headerlink" title="TL;DR"></a>TL;DR</h1><p>抓包是常用的问题排查手段,通过获取网络通讯内容的方式分析应用工作情况,辅助定位问题。</p>
<p>抓包工具相当多,PC/服务端常见的抓包工具 <code>tcpdump</code>/<code>Wireshark</code>/<code>Fiddler</code>/<code>Charles</code>/<code>whistle</code>/<code>mitmproxy</code> 。</p>
<p>本篇只做简要介绍,具体工具使用再起篇幅。</p>
<!-- packet-capture-i -->
<a id="more"></a>
<h1 id="工具"><a href="#工具" class="headerlink" title="工具"></a>工具</h1><h2 id="对比"><a href="#对比" class="headerlink" title="对比"></a>对比</h2><p>个人对之前提及的工具做一个简单的对比:</p>
<table>
<thead>
<tr>
<th>特性/软件</th>
<th>tcpdump</th>
<th>Wireshark</th>
<th>Fiddler</th>
<th>Charles</th>
<th>whistle</th>
<th>mitmproxy</th>
</tr>
</thead>
<tbody>
<tr>
<td><strong>HTTP(S)协议展现效果</strong></td>
<td>无</td>
<td>一般</td>
<td>好</td>
<td>好</td>
<td>好</td>
<td>好</td>
</tr>
<tr>
<td><strong>TCP/UDP协议展现效果</strong></td>
<td>一般</td>
<td>好</td>
<td>不支持</td>
<td>不支持</td>
<td>不支持</td>
<td>不支持</td>
</tr>
<tr>
<td><strong>协议支持</strong></td>
<td>多</td>
<td>多</td>
<td>主要支持HTTP(S)</td>
<td>主要支持HTTP(S)</td>
<td>主要支持HTTP(S)</td>
<td>主要支持HTTP(S)</td>
</tr>
<tr>
<td><strong>安装配置</strong></td>
<td>简单</td>
<td>简单</td>
<td>简单</td>
<td>简单</td>
<td>一般</td>
<td>一般</td>
</tr>
<tr>
<td><strong>支持平台</strong></td>
<td>主要是*nix</td>
<td>Windows/Mac/Linux</td>
<td>Windows(其他平台需要Mono)</td>
<td>Windows/Mac/Linux</td>
<td>基于node.js</td>
<td>基于Python</td>
</tr>
<tr>
<td><strong>费用</strong></td>
<td>免费</td>
<td>免费</td>
<td>免费</td>
<td>收费</td>
<td>免费</td>
<td>免费</td>
</tr>
<tr>
<td><strong>GUI</strong></td>
<td>无</td>
<td>有</td>
<td>有</td>
<td>有</td>
<td>有(基于浏览器)</td>
<td>有(基于浏览器)</td>
</tr>
<tr>
<td><strong>源代码</strong></td>
<td>公开</td>
<td>公开</td>
<td>不公开</td>
<td>不公开</td>
<td>公开</td>
<td>公开</td>
</tr>
</tbody>
</table>
<h1 id="参考"><a href="#参考" class="headerlink" title="参考"></a>参考</h1><ul>
<li><a href="https://github.com/avwo/whistle" target="_blank" rel="external">whistle</a></li>
<li><a href="https://www.tcpdump.org/manpages/tcpdump.1.html" target="_blank" rel="external">tcpdump</a></li>
<li><a href="https://www.jianshu.com/p/ed6db49a3428" target="_blank" rel="external">libpcap实现机制及接口函数</a></li>
<li><a href="https://blog.devtang.com/2015/11/14/charles-introduction/" target="_blank" rel="external">Charles 从入门到精通| 唐巧的博客</a></li>
</ul>
<h1 id="TL-DR"><a href="#TL-DR" class="headerlink" title="TL;DR"></a>TL;DR</h1><p>抓包是常用的问题排查手段,通过获取网络通讯内容的方式分析应用工作情况,辅助定位问题。</p>
<p>抓包工具相当多,PC/服务端常见的抓包工具 <code>tcpdump</code>/<code>Wireshark</code>/<code>Fiddler</code>/<code>Charles</code>/<code>whistle</code>/<code>mitmproxy</code> 。</p>
<p>本篇只做简要介绍,具体工具使用再起篇幅。</p>
<!-- packet-capture-i -->
Java自定义注解(使用篇)
http://www.liaoaoyang.com/articles/2018/10/24/java-annotation/
2018-10-24T15:53:01.000Z
2019-01-18T16:47:16.983Z
<h1 id="TL-DR"><a href="#TL-DR" class="headerlink" title="TL;DR"></a>TL;DR</h1><p>Java 注解广泛运用在开发之中,用于增强变量/方法/类等。</p>
<p>尝试说明 Java 自定义注解的使用,以及通过开源项目中的使用进行说明。</p>
<!-- java-aonnotaion -->
<a id="more"></a>
<p>本文主要记录个人的理解,全文基于Java SE8。</p>
<h1 id="自定义注解"><a href="#自定义注解" class="headerlink" title="自定义注解"></a>自定义注解</h1><p>自定义注解分为两个部分:<code>注解声明</code>和<code>注解处理逻辑</code>。</p>
<p>每个注解可以有多个属性值,同名注解通过声明后可以在对象上使用多个。</p>
<h2 id="注解结构"><a href="#注解结构" class="headerlink" title="注解结构"></a>注解结构</h2><h3 id="定义注解"><a href="#定义注解" class="headerlink" title="定义注解"></a>定义注解</h3><p>用以下实例说明:</p>
<figure class="highlight dart"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"><span class="annotation">@Repeatable</span>(LearnRepeatableAnnotation.<span class="keyword">class</span>)</span><br><span class="line"><span class="annotation">@Retention</span>(RetentionPolicy.RUNTIME)</span><br><span class="line"><span class="annotation">@Target</span>({ElementType.FIELD, ElementType.LOCAL_VARIABLE, ElementType.METHOD})</span><br><span class="line">public <span class="annotation">@interface</span> LearnAnnotation {</span><br><span class="line"> <span class="built_in">String</span> value() <span class="keyword">default</span> <span class="string">""</span>;</span><br><span class="line"></span><br><span class="line"> <span class="built_in">String</span> filedAnnotationValue() <span class="keyword">default</span> <span class="string">""</span>;</span><br><span class="line"></span><br><span class="line"> Class<?> className() <span class="keyword">default</span> Void.<span class="keyword">class</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<p>逐行分析一下。</p>
<p><code>@Repeatable(LearnRepeatableAnnotation.class)</code> 表示本注解可以在一个对象上使用多次,具体内容下文会具体说明。</p>
<p><code>@Retention(RetentionPolicy.RUNTIME)</code> 是一个元注解,表示注解可以在运行时通过反射使用,元注解下文会具体说明。</p>
<p><code>@Target({ElementType.FIELD, ElementType.LOCAL_VARIABLE, ElementType.METHOD})</code> 也是一个元注解,表示注解可以在属性、本地变量、方法上。</p>
<p><code>public @interface LearnAnnotation</code> 表示这是一个注解声明,注解需要以<code>@interface</code>声明。</p>
<p><code>String value() default "";</code> 表示注解的值域是字符串类型,默认为空字符串。注解使用时,可以通过<code>属性名=值</code>的形式进行赋值,如果不声明属性名,说明会赋值到<code>value</code>属性上。注解中的属性名就是声明中的方法名。</p>
<p><code>String filedAnnotationValue() default "";</code> 表示自定义注解<code>@LearnAnnotation</code>有一个名为<code>filedAnnotationValue</code>的字符串属性,使用时可以通过<code>@LearnAnnotation(filedAnnotationValue = "NAME")</code>这一形式使用。</p>
<p><code>Class<?> className() default Void.class;</code> 表示自定义注解<code>@LearnAnnotation</code>有一个名为<code>className</code>的Class对象,此处需要注意,自定义注解的属性值只能是基本类型(<code>short</code>/<code>int</code>/<code>long</code>/<code>boolean</code>/<code>char</code>/<code>String</code>/<code>enum</code>/<code>Class</code>等)以及他们这些类型的数组。</p>
<p>注解如果没有default声明的,需要指定属性值后才能使用。</p>
<h3 id="同一对象使用多个相同的注解声明"><a href="#同一对象使用多个相同的注解声明" class="headerlink" title="同一对象使用多个相同的注解声明"></a>同一对象使用多个相同的注解声明</h3><p>还是使用上述案例,第一行<code>@Repeatable(LearnRepeatableAnnotation.class)</code>通过声明利用<code>@LearnRepeatableAnnotation</code>这一注解表示可以在一个对象上使用多个<code>LearnAnnotation</code>注解。</p>
<p><code>@LearnRepeatableAnnotation</code> 的实现如下:</p>
<figure class="highlight less"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="variable">@Retention</span>(RetentionPolicy.RUNTIME)</span><br><span class="line"><span class="variable">@Target</span>({ElementType.FIELD, ElementType.LOCAL_VARIABLE, ElementType.METHOD})</span><br><span class="line">public <span class="variable">@interface</span> LearnRepeatableAnnotation {</span><br><span class="line"> <span class="tag">LearnAnnotation</span><span class="attr_selector">[]</span> <span class="tag">value</span>() <span class="tag">default</span> {};</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<p>即需要声明一个新类型的注解,且这一注解的值,是计划使用多个注解的数组。</p>
<p>实际使用中可以像如下形式使用:</p>
<figure class="highlight less"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="variable">@LearnAnnotation</span>(filedAnnotationValue = <span class="string">"v1"</span>)</span><br><span class="line"><span class="variable">@LearnAnnotation</span>(value = <span class="string">"v2"</span>)</span><br><span class="line">private int testRepeatInt = <span class="number">0</span>;</span><br></pre></td></tr></table></figure>
<p>使用多个同名注解,例如作为配置规则,可以让当前对象获取多个规则。</p>
<h2 id="注解声明"><a href="#注解声明" class="headerlink" title="注解声明"></a>注解声明</h2><p>注解声明又主要分为两个部分:元注解注解名称及字段定义。</p>
<h3 id="元注解"><a href="#元注解" class="headerlink" title="元注解"></a>元注解</h3><p>Java 中提供了4种元注解:</p>
<ul>
<li>@Documented - 在JavaDoc中提供注解信息</li>
<li>@Retention - 注解的生效范围</li>
<li>@Target - 注解允许使用的对象</li>
<li>@Inherited - 注解是否可以被子类继承</li>
</ul>
<p>元注解是实现自定义注解的重要工具,最重要的是<code>@Retention</code>与<code>@Target</code>。</p>
<h4 id="元注解-Retention"><a href="#元注解-Retention" class="headerlink" title="元注解@Retention"></a>元注解@Retention</h4><p>元注解 <code>@Retention</code> 可以有如下3个属性值:</p>
<ul>
<li><code>RetentionPolicy.SOURCE</code> – 注解保留在源码中,编译阶段会被丢弃</li>
<li><code>RetentionPolicy.CLASS</code> – 注解保留在<code>.class</code>文件中,但不会在运行时存在</li>
<li><code>RetentionPolicy.RUNTIME</code> – 注解可以在运行时读取、使用反射可以获得</li>
</ul>
<p>默认是<code>RetentionPolicy.CLASS</code>。</p>
<p>在注解指定了 <code>RetentionPolicy.SOURCE</code> 的情况下,Java 编译源码时,会有一个注解处理阶段,会调用对应的注解处理器(继承自<code>javax.annotation.processing.AbstractProcessor</code>的子类),在编译时,可以执行注解的一些逻辑,例如<a href="https://github.com/liaoaoyang/LearningJava/blob/master/spring-study/src/main/java/co/iay/learn/learningjava/spring/springstudy/annotation/LJTransferProcessor.java" target="_blank" rel="external">生成代码</a>。</p>
<p>在 IDEA 中,如果是 maven 项目,可以通过项目配置指定注解处理器,也可以通过 <code>maven-processor-plugin</code> <a href="https://github.com/liaoaoyang/LearningJava/blob/master/spring-study/pom.xml" target="_blank" rel="external">实现</a>。</p>
<h4 id="元注解-Target"><a href="#元注解-Target" class="headerlink" title="元注解@Target"></a>元注解@Target</h4><p>元注解 <code>@Target</code> 可以有如下8个属性值:</p>
<ul>
<li><code>ElementType.ANNOTATION_TYPE</code></li>
<li><code>ElementType.CONSTRUCTOR</code></li>
<li><code>ElementType.FIELD</code></li>
<li><code>ElementType.LOCAL_VARIABLE</code></li>
<li><code>ElementType.METHOD</code></li>
<li><code>ElementType.PACKAGE</code></li>
<li><code>ElementType.PARAMETER</code></li>
<li><code>ElementType.TYPE</code></li>
</ul>
<p>作用范围看名称基本都能对应上。默认是全部。</p>
<h1 id="开源项目中的使用"><a href="#开源项目中的使用" class="headerlink" title="开源项目中的使用"></a>开源项目中的使用</h1><h2 id="Fastjson"><a href="#Fastjson" class="headerlink" title="Fastjson"></a>Fastjson</h2><p><a href="https://github.com/alibaba/fastjson" target="_blank" rel="external">Fastjson</a> 作为广泛使用的 Java JSON 解析类库,广泛应用了注解。</p>
<h3 id="字段别名"><a href="#字段别名" class="headerlink" title="字段别名"></a>字段别名</h3><p>POJO 中如果使用驼峰命名,但是 API 中需要使用下划线分隔,这样的场景并不少见。</p>
<p>Fastjson 中通过 <code>@JSONField</code> 注解可以实现这一功能。</p>
<p>Fastjson 通过 <code>toJSONString()</code> 方法实现对象转化到 JSON 格式字符串的行为,首先会根据转换对象的类型解析出对象各个字段的信息(参见<code>com.alibaba.fastjson.util.FieldInfo</code>),读取每个字段上的 <code>@JSONField</code> 注解,在转化为字符串过程中,当需要写入键时,如果注解 <code>name</code> 值存在,则写入 <code>name</code> 配置的值。</p>
<h2 id="Spring"><a href="#Spring" class="headerlink" title="Spring"></a>Spring</h2><h3 id="AOP"><a href="#AOP" class="headerlink" title="AOP"></a>AOP</h3><p>AOP 是常用的编程模式。</p>
<p>在 Spring 中定义一个 Aspect,通过 <code>@Pointcut</code> 注解关联何种对象的何种操作,通过 <code>@Before</code>/<code>@After</code> 等注解指定当前 Aspect 各个阶段可以执行的方法(Advice)。</p>
<p>Spring 框架在扫描各个 bean 时,会根据指定的 Aspect 信息,为各个 bean 的指定方法关联上各个 Advice,在执行时逐个运行。</p>
<p>例如对外提供接口调用时,需要对一些接口提供自定的参数校验等功能,可以考虑通过自定义注解的方式,提供一个 <code>@Around</code> advice,判断参数是否合理即可。</p>
<h1 id="Lombok"><a href="#Lombok" class="headerlink" title="Lombok"></a>Lombok</h1><p>Java 开发中对象的 Getter/Setter 方法以及常规的构造方法让代码变得臃肿,<a href="https://github.com/rzwitserloot/lombok" target="_blank" rel="external">Lombok</a> 通过注解的方式,在编译阶段修改 AST,实现生成的 class 文件中带有对应的方法。</p>
<p>以上内容还可以另起篇幅详细说明。</p>
<h1 id="参考"><a href="#参考" class="headerlink" title="参考"></a>参考</h1><ul>
<li><a href="https://docs.oracle.com/javase/tutorial/java/annotations/index.html" target="_blank" rel="external">Lesson: Annotations (Oracle Java Documentation)</a></li>
<li><a href="https://medium.com/@iammert/annotation-processing-dont-repeat-yourself-generate-your-code-8425e60c6657" target="_blank" rel="external">Annotation Processing : Don’t Repeat Yourself, Generate Your Code.</a></li>
<li><a href="http://notatube.blogspot.com/2010/11/project-lombok-trick-explained.html" target="_blank" rel="external">Project Lombok - Trick Explained</a></li>
</ul>
<h1 id="TL-DR"><a href="#TL-DR" class="headerlink" title="TL;DR"></a>TL;DR</h1><p>Java 注解广泛运用在开发之中,用于增强变量/方法/类等。</p>
<p>尝试说明 Java 自定义注解的使用,以及通过开源项目中的使用进行说明。</p>
<!-- java-aonnotaion -->
Java C1000K 笔记
http://www.liaoaoyang.com/articles/2018/09/10/java-c1000k-notes/
2018-09-10T15:52:54.000Z
2018-11-03T21:38:32.118Z
<h1 id="TL-DR"><a href="#TL-DR" class="headerlink" title="TL;DR"></a>TL;DR</h1><p>Java 实现 C1000K 需要对服务器进行一定的调整,同时也需要选择合理的编程方式。</p>
<p>个人对 Java 实现 C1000K 的学习笔记。虽然早已不是难事,终须自己实现一遍。</p>
<a id="more"></a>
<!-- java-c1000k-notes -->
<h1 id="前提"><a href="#前提" class="headerlink" title="前提"></a>前提</h1><h2 id="C1000K-or-1000K-QPS?"><a href="#C1000K-or-1000K-QPS?" class="headerlink" title="C1000K or 1000K QPS?"></a>C1000K or 1000K QPS?</h2><p>首先需要说明,C1000K 并不指的是单机 1000W QPS。</p>
<p>并发的连接数不全是活跃连接。</p>
<h2 id="服务器"><a href="#服务器" class="headerlink" title="服务器"></a>服务器</h2><p>8核16G ECS,操作系统CentOS 7。</p>
<h1 id="配置"><a href="#配置" class="headerlink" title="配置"></a>配置</h1><h2 id="内核参数"><a href="#内核参数" class="headerlink" title="内核参数"></a>内核参数</h2><h3 id="文件"><a href="#文件" class="headerlink" title="文件"></a>文件</h3><p>并发百万连接早已不是难事,但是服务器默认的一些配置仍然需要配置,默认情况下的配置并不足以支撑这一需求。</p>
<p>Linux 一切皆文件,Socket 连接也是文件。</p>
<p>内核对可以打开的文件数做了限制,默认较小,肯定无法达到百万量级。不调整就使用,会提示 <code>Too many open files</code>,这个问题在 ES 的使用过程中很容易遇到,同学启动集群时不做任何内核参数配置,索引增多之后无法建立连接查询ES,会在错误日志中看到相关的错误信息。</p>
<p>日常使用 <code>ulimit</code> 进行设置,在内核限制的情况下,即便设置成功页只对当前会话终端有效。</p>
<p>在 <code>/etc/sysctl.conf</code> 中配置文件打开的内核参数,在 <code>/etc/security/limits.conf</code> 中配置进程可以打开的文件参数。</p>
<p>主要限制文件打开数的有如下内核参数:</p>
<ul>
<li>fs.file-max</li>
<li>fs.nr_open</li>
</ul>
<p>关于二者的定义可以在<a href="https://www.kernel.org/doc/Documentation/sysctl/fs.txt" target="_blank" rel="external">内核文档</a>上查到:</p>
<figure class="highlight applescript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br></pre></td><td class="code"><pre><span class="line">==============================================================</span><br><span class="line"></span><br><span class="line"><span class="type">file</span>-max & <span class="type">file</span>-nr:</span><br><span class="line"></span><br><span class="line">The value <span class="keyword">in</span> <span class="type">file</span>-max denotes <span class="keyword">the</span> maximum <span class="type">number</span> <span class="keyword">of</span> <span class="type">file</span>-</span><br><span class="line">handles <span class="keyword">that</span> <span class="keyword">the</span> Linux kernel will allocate. When you <span class="keyword">get</span> lots</span><br><span class="line"><span class="keyword">of</span> <span class="keyword">error</span> messages <span class="keyword">about</span> <span class="property">running</span> <span class="keyword">out of</span> <span class="type">file</span> handles, you might</span><br><span class="line">want <span class="keyword">to</span> increase this limit.</span><br><span class="line"></span><br><span class="line">Historically,<span class="keyword">the</span> kernel was able <span class="keyword">to</span> allocate <span class="type">file</span> handles</span><br><span class="line">dynamically, <span class="keyword">but</span> <span class="keyword">not</span> <span class="keyword">to</span> free them again. The three values <span class="keyword">in</span></span><br><span class="line"><span class="type">file</span>-nr denote <span class="keyword">the</span> <span class="type">number</span> <span class="keyword">of</span> allocated <span class="type">file</span> handles, <span class="keyword">the</span> <span class="type">number</span></span><br><span class="line"><span class="keyword">of</span> allocated <span class="keyword">but</span> unused <span class="type">file</span> handles, <span class="keyword">and</span> <span class="keyword">the</span> maximum <span class="type">number</span> <span class="keyword">of</span></span><br><span class="line"><span class="type">file</span> handles. Linux <span class="number">2.6</span> always reports <span class="number">0</span> <span class="keyword">as</span> <span class="keyword">the</span> <span class="type">number</span> <span class="keyword">of</span> free</span><br><span class="line"><span class="type">file</span> handles <span class="comment">-- this is not an error, it just means that the</span></span><br><span class="line"><span class="type">number</span> <span class="keyword">of</span> allocated <span class="type">file</span> handles exactly matches <span class="keyword">the</span> <span class="type">number</span> <span class="keyword">of</span></span><br><span class="line">used <span class="type">file</span> handles.</span><br><span class="line"></span><br><span class="line">Attempts <span class="keyword">to</span> allocate more <span class="type">file</span> descriptors than <span class="type">file</span>-max are</span><br><span class="line">reported <span class="keyword">with</span> printk, look <span class="keyword">for</span> <span class="string">"VFS: file-max limit <number></span><br><span class="line">reached"</span>.</span><br><span class="line">==============================================================</span><br><span class="line"></span><br><span class="line">nr_open:</span><br><span class="line"></span><br><span class="line">This denotes <span class="keyword">the</span> maximum <span class="type">number</span> <span class="keyword">of</span> <span class="type">file</span>-handles a process can</span><br><span class="line">allocate. Default value <span class="keyword">is</span> <span class="number">1024</span>*<span class="number">1024</span> (<span class="number">1048576</span>) which should be</span><br><span class="line">enough <span class="keyword">for</span> most machines. Actual limit depends <span class="function_start"><span class="keyword">on</span></span> RLIMIT_NOFILE</span><br><span class="line">resource limit.</span><br><span class="line"></span><br><span class="line">==============================================================</span><br></pre></td></tr></table></figure>
<p>二者的定义是 <code>fs.file-max</code> 决定了内核可以打开的文件数量(不同于文件描述符,这里指的是指向实际文件结构的对象,<a href="http://www.linuxvox.com/post/what-are-file-max-and-file-nr-linux-kernel-parameters/" target="_blank" rel="external">参见</a> ),<code>fs.nr_open</code> 决定了单个进程可以打开的最大文件数量。但是一般对应文件对象的都具有一个文件描述符,所以姑且可以认为二者数目相等。</p>
<p>再看 ECS 默认的 <code>/etc/security/limits.conf</code>:</p>
<figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">root soft nofile <span class="number">65535</span></span><br><span class="line">root hard nofile <span class="number">65535</span></span><br><span class="line">* soft nofile <span class="number">65535</span></span><br><span class="line">* hard nofile <span class="number">65535</span></span><br></pre></td></tr></table></figure>
<p>soft 和 hard 的区别是 hard 值表示是参数的最大值,soft 值为设定值,在 hard 值的范围以内可以随意修改。</p>
<p>此处需要注意,如果 <code>/etc/security/limits.conf</code> 设定的值大于 <code>fs.nr_open</code> 会引起无法登录服务器的问题,一旦修改错误,如果还没有断开连接,立即调低当前的值,使之小于 <code>fs.nr_open</code>,否则,只能进入单用户模式恢复了。</p>
<p>综上,<code>fs.file-max</code> >= <code>fs.nr_open</code> >= <code>configs in /etc/security/limits.conf</code>。</p>
<p>服务端配置如上参数,压测客户端也需要配置,否则无法模拟出大量连接。</p>
<h3 id="网络"><a href="#网络" class="headerlink" title="网络"></a>网络</h3><h4 id="服务端"><a href="#服务端" class="headerlink" title="服务端"></a>服务端</h4><p>Echo Server 使用 TCP 连接,服务端需要先 bind 一个端口,之后 accept 新连接。</p>
<p>TCP 三次握手无需多言:</p>
<p>1) 客户端发出 <code>SYN_(a)</code><br>2) 服务端收到 <code>SYN_(a)</code>,服务端返回 <code>SYN_(b) + ACK(a + 1)</code><br>3) 客户端收到 <code>SYN_(b) + ACK(a + 1)</code>,客户端返回 <code>ACK(b + 1)</code></p>
<p>在步骤<code>1</code>之后本次网络连接就是半连接状态,会进入一个队列,这个队列的大小由内核参数 <code>net.ipv4.tcp_max_syn_backlog</code> 决定。</p>
<p>在步骤<code>3</code>之后,会将连接放入Accept队列,队列的大小由内核参数 <code>net.core.somaxconn</code> 决定。</p>
<p>这两个队列至关重要,只有调用 <code>accept()</code> 方法之后,整个连接才是可以收发数据的状态,如果长时间不调用 <code>accept()</code>,客户端已经认为连接建立,发送数据会出现 <code>client fooling</code> 问题,长时间得不到回应会进入重试,一段时间过后客户端会主动发 <code>FIN</code> 断开连接,导致连接不成功。</p>
<p>以上问题的详情可以阅读博文 <a href="http://jm.taobao.org/2017/05/25/525-1/" target="_blank" rel="external">《关于TCP 半连接队列和全连接队列》</a> 了解更多。</p>
<p>回到 Echo Server 上,由于 <code>accept()</code> 操作不能马上完成,需要一定时间,在突发请求,或者压测的情况下瞬间建立大量连接,会导致队列拥塞,最后仍然会引起无法建立连接的问题。所以,适度提高队列大小有助于 Echo Server 以及实际网络服务器的开发。</p>
<figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="preprocessor"># 最大值,原因参见 https:<span class="comment">//stackoverflow.com/questions/23862410/invalid-argument-setting-key-net-core-somaxconn</span></span></span><br><span class="line">net.core.somaxconn = <span class="number">65535</span></span><br><span class="line">net.ipv4.tcp_max_syn_backlog = <span class="number">65535</span></span><br></pre></td></tr></table></figure>
<p><code>net.core.somaxconn</code> 这个参数还会影响应用代码中 <code>backlog</code> 的取值,取值为 <code>min(net.core.somaxconn, backlog)</code>,这部分在服务端编码中再细致说明。</p>
<p>如果开启了 <code>iptables</code>,还需要注意 <code>net.*.nf_conntrack_max</code> 参数需要超过 1000K。</p>
<h4 id="客户端"><a href="#客户端" class="headerlink" title="客户端"></a>客户端</h4><p>作为压测客户端,需要注意的是,TCP请求可以看做一个四元组:</p>
<p><code>客户端IP-客户端端口-服务端IP-服务端口</code></p>
<p>想要获得大量的客户端连接,首先就需要足够多的端口。</p>
<p>端口范围的内核参数 <code>net.ipv4.ip_local_port_range</code> 默认值范围不大,大约在30000个左右,我们可以将其放宽到保留端口附近的范围,这样,就能产生超过60000个客户端连接了。</p>
<h1 id="开发"><a href="#开发" class="headerlink" title="开发"></a>开发</h1><h2 id="客户端-1"><a href="#客户端-1" class="headerlink" title="客户端"></a>客户端</h2><p>考虑到开发的难度以及趣味性,选择 Go 进行开发(代码见<a href="https://github.com/liaoaoyang/LearningJava/blob/master/scripts/simple_nio_test_echo_client.go" target="_blank" rel="external">GitHub</a>)。</p>
<p>核心内容如下:</p>
<ul>
<li>每个请求产生一个协程,模拟一个客户端</li>
<li>并发程度通过 channel 控制,由于 channel 可以阻塞住操作,可以通过产生与并发度相同大小的队列,当一个请求完成时读取一个数据,起到控制并发度的效果</li>
<li>同样通过 channel 完成请求数的计数操作</li>
</ul>
<p>考虑到单网卡难以模拟出百万级别连接(单网卡对应服务端端口只能使用约60000个端口),可以使用 docker 模拟出 16+ 客户端访问 <code>docker0</code> 设备上侦听端口的服务端程序,或者选择服务端开启 16 个以上端口。</p>
<h2 id="服务端-1"><a href="#服务端-1" class="headerlink" title="服务端"></a>服务端</h2><p>服务端不会主动断开连接,TIME-WAIT 问题暂且不用处理。</p>
<p>通过 BIO + 多线程模式在客户端数量较少时没有问题,Java 目前了解到一个用户线程对应一个内核线程,客户端数目多时,为了降低系统消耗,考虑使用 NIO 实现(代码见<a href="https://github.com/liaoaoyang/LearningJava/blob/master/src/main/java/co/iay/learn/learningjava/nio/SimpleNIOEchoServerMT.java" target="_blank" rel="external">GitHub</a>)。</p>
<p>NIO 的核心就是一个线程管理多个连接,通过 Selector 实现对多个连接读写事件的监听,在读写时间触发时处理 IO 操作。</p>
<p>选用 Java NIO 实现 C1000K 服务器,实际上是 Reactor 模式的具体实现。</p>
<p>为了提高处理速度,考虑将 accept 操作与 IO 操作分开。</p>
<p>IO 操作即便是在非阻塞的 Channel 里也是占用 CPU 时间的,为了充分利用多核心,可以考虑将线程数调成CPU个数加一个。</p>
<h1 id="效果"><a href="#效果" class="headerlink" title="效果"></a>效果</h1><p>服务端选择启动多个端口:</p>
<figure class="highlight nginx"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="title">java</span> -jar -Dbacklog=<span class="number">40000</span> -Dhostname=<span class="number">192.168.16.1</span> -Dport=`seq <span class="number">30001</span> <span class="number">30020</span>|tr <span class="string">"\n"</span> <span class="string">","</span>|sed <span class="string">'s/,$//'</span>` SimpleNIOEchoServerMT.jar</span><br></pre></td></tr></table></figure>
<p>客户端选择压测多个端口,并使用长连接,每 30+rand(0,30) 秒发送一个数据包,平均每秒活跃连接数30000左右:</p>
<figure class="highlight stata"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">go <span class="keyword">run</span> simple_nio_test_echo_client.go -<span class="keyword">h</span>=192.168.16.1 -P=`seq 30001 30020|tr <span class="string">"\n"</span> <span class="string">","</span>|sed 's/,$<span class="comment">//'` -i 30 -r 10 -c 60000</span></span><br></pre></td></tr></table></figure>
<p>使用 ss 查看:</p>
<figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">➜ ~ ss -s</span><br><span class="line">Total: <span class="number">1153054</span> (kernel <span class="number">1153152</span>)</span><br><span class="line">TCP: <span class="number">1174832</span> (estab <span class="number">1055390</span>, closed <span class="number">92005</span>, orphaned <span class="number">22030</span>, synrecv <span class="number">0</span>, timewait <span class="number">0</span>/<span class="number">0</span>), ports <span class="number">0</span></span><br><span class="line"></span><br><span class="line">Transport Total IP IPv6</span><br><span class="line">* <span class="number">1153152</span> - -</span><br><span class="line">RAW <span class="number">0</span> <span class="number">0</span> <span class="number">0</span></span><br><span class="line">UDP <span class="number">9</span> <span class="number">8</span> <span class="number">1</span></span><br><span class="line">TCP <span class="number">1082827</span> <span class="number">1082827</span> <span class="number">0</span></span><br><span class="line">INET <span class="number">1082836</span> <span class="number">1082835</span> <span class="number">1</span></span><br><span class="line">FRAG <span class="number">0</span> <span class="number">0</span> <span class="number">0</span></span><br></pre></td></tr></table></figure>
<p>以上是 NIO 实现 C1000K 服务器的摘要。</p>
<h1 id="参考"><a href="#参考" class="headerlink" title="参考"></a>参考</h1><ul>
<li><a href="http://blog.yufeng.info/archives/category/network/" target="_blank" rel="external">强烈推荐余锋老师的博文</a></li>
<li><a href="https://www.cnxct.com/coping-with-the-tcp-time_wait-state-on-busy-linux-servers-in-chinese-and-dont-enable-tcp_tw_recycle/" target="_blank" rel="external">不要在linux上启用net.ipv4.tcp_tw_recycle参数</a></li>
<li><a href="http://jm.taobao.org/2017/05/25/525-1/" target="_blank" rel="external">关于TCP 半连接队列和全连接队列</a></li>
<li><a href="http://www.ideawu.net/blog/archives/740.html" target="_blank" rel="external">构建C1000K的服务器(1)</a></li>
<li><a href="https://awen.me/post/59062.html" target="_blank" rel="external">iptables 的 conntrack 连接跟踪模块</a></li>
<li><a href="https://www.jianshu.com/p/e0b52dc702d6" target="_blank" rel="external">Node版单机100w连接(C1000K)是如何达成的</a></li>
</ul>
<h1 id="TL-DR"><a href="#TL-DR" class="headerlink" title="TL;DR"></a>TL;DR</h1><p>Java 实现 C1000K 需要对服务器进行一定的调整,同时也需要选择合理的编程方式。</p>
<p>个人对 Java 实现 C1000K 的学习笔记。虽然早已不是难事,终须自己实现一遍。</p>
Redis中的Lua
http://www.liaoaoyang.com/articles/2018/08/02/lua-in-redis/
2018-08-02T15:42:23.000Z
2018-08-07T13:59:03.248Z
<h1 id="TL-DR"><a href="#TL-DR" class="headerlink" title="TL;DR"></a>TL;DR</h1><p>Redis 中使用 Lua 的相关笔记。</p>
<!-- lua-in-redis -->
<a id="more"></a>
<h1 id="原理"><a href="#原理" class="headerlink" title="原理"></a>原理</h1><p>Lua 这门语言的运行时库大小相当之小,可以嵌入由 C 语言实现的应用程序之中,动态的执行功能。</p>
<p>具体设计与实现在文档中十分清晰,传送门:</p>
<ul>
<li><a href="http://redisbook.readthedocs.io/en/latest/feature/scripting.html" target="_blank" rel="external">http://redisbook.readthedocs.io/en/latest/feature/scripting.html</a></li>
<li><a href="http://redisdoc.com/script/eval.html" target="_blank" rel="external">http://redisdoc.com/script/eval.html</a></li>
</ul>
<p>简而言之,即 Redis 集成了 Lua 的运行时库,加载部分 Lua 的基本库函数,通过伪客户端完成脚本与数据库的交互,转换数据格式。</p>
<h1 id="Redis-Lua-适合做什么"><a href="#Redis-Lua-适合做什么" class="headerlink" title="Redis + Lua 适合做什么"></a>Redis + Lua 适合做什么</h1><p>个人理解,这个组合赋予了 Redis 更强大的服务端计算能力,适合如下特点的任务:</p>
<ol>
<li>带有特定判断逻辑的操作</li>
<li>耗时低的操作</li>
</ol>
<h2 id="为什么适合带有特定判断逻辑的操作?"><a href="#为什么适合带有特定判断逻辑的操作?" class="headerlink" title="为什么适合带有特定判断逻辑的操作?"></a>为什么适合带有特定判断逻辑的操作?</h2><h3 id="原子操作里的逻辑判断"><a href="#原子操作里的逻辑判断" class="headerlink" title="原子操作里的逻辑判断"></a>原子操作里的逻辑判断</h3><p>Redis 的原子性在 <a href="https://liaoaoyang.cn/articles/2017/08/06/interesting-and-useful-designs-of-redis/" target="_blank" rel="external">Redis里那些有用又有趣的设计</a> 一文中有过讨论,单进程单线程按序执行(即按客户端可读可写事件触发事件处理顺序)保证了原子性。</p>
<p>Redis 本身也可以通过 MULTI/EXEC 等命令实现一组原子的操作。</p>
<p>然而,需要注意的是,MULTI 是在所有指令发送完成之后,一次性执行,在执行的过程中,客户端是没有办法及时获得反馈的。如果需要根据某一指令的操作结果,决定下一步执行何种操作,是无法实现的。</p>
<p>Lua 脚本则可以带有判断逻辑,在客户端指令发出后,根据入参可以控制指令的运行分支。</p>
<h3 id="CAS"><a href="#CAS" class="headerlink" title="CAS"></a>CAS</h3><p>Redis 的 CAS 操作可以通过 WATCH 来实现,但是也可以直接通过 Lua 脚本的形式完成,如果还想根据操作的结果完成其他工作,Lua 脚本可能就更为适合了。</p>
<p>希望对事务进行干预的操作,可以考虑使用 Lua 脚本完成。</p>
<h2 id="为什么适合耗时低的操作"><a href="#为什么适合耗时低的操作" class="headerlink" title="为什么适合耗时低的操作"></a>为什么适合耗时低的操作</h2><p>Redis 的运行模型,决定了强计算任务会阻塞住其他指令的执行,在通过 Lua 扩展功能时也需要注意。</p>
<p>Redis在执行 Lua 脚本时,最大执行时间的长短由 lua-time-limit 选项来控制(以毫秒为单位),可以通过编辑 redis.conf 文件或者使用 CONFIG GET 和 CONFIG SET 命令来修改它。</p>
<p>当执行超时时,只是重新开始接受新的操作,但是只能处理 <code>SCRIPT KILL</code> 和 <code>SHUTDOWN NOSAVE</code> 两个命令。如果已经执行了写操作,只能通过 <code>SHUTDOWN NOSAVE</code> 重启服务器完成工作了。</p>
<p>所以,从性能和安全角度出发,短小精悍的 Lua 脚本是值得推荐的。</p>
<h1 id="如何使用"><a href="#如何使用" class="headerlink" title="如何使用"></a>如何使用</h1><p>在 Redis 中通过 Lua 扩展自身的功能,可以有两种方式:直接提供脚本,先导入后使用。</p>
<p>以 <a href="https://github.com/liaoaoyang/toolbox/blob/master/softwares/redis/incrby_not_over.lua" target="_blank" rel="external">incrby_not_over.lua</a> 为例:</p>
<p>既可以通过:</p>
<figure class="highlight autoit"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">redis-cli <span class="built_in">eval</span> <span class="string">"$(cat incrby_not_over.lua)"</span> <span class="number">1</span> key_name max_value [<span class="keyword">step</span> expire]</span><br></pre></td></tr></table></figure>
<p>也可以:</p>
<figure class="highlight accesslog"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">redis-cli evalsha xxx <span class="number">1</span> key_name max_value <span class="string">[step expire]</span></span><br></pre></td></tr></table></figure>
<p>当然,在使用 <a href="http://redisdoc.com/script/eval.html" target="_blank" rel="external">EVAL</a> 命令一次之后,可以直接通过 <a href="http://redisdoc.com/script/evalsha.html" target="_blank" rel="external">EVALSHA</a> 结合当前脚本的 sha1 值进行调用。</p>
<p>执行过一次 EVAL 和 <a href="http://redisdoc.com/script/script_load.html" target="_blank" rel="external">SCRIPT LOAD</a> 的效果基本是一致的,后者不会实际执行脚本。</p>
<p>可以通过 <a href="http://redisdoc.com/script/script_exists.html" target="_blank" rel="external">SCRIPT EXISTS</a> 指令判断特定的 sha1 值对应的脚本是否存在。</p>
<p>在 Redis 重启之后,脚本需要重新导入。</p>
<h1 id="脚本-sha1-值的生成方式"><a href="#脚本-sha1-值的生成方式" class="headerlink" title="脚本 sha1 值的生成方式"></a>脚本 sha1 值的生成方式</h1><p>想要迅速求得 Lua 脚本(假设文件名为<code>script.lua</code>)导入之后 sha1 值,可以在 Shell 中使用如下方法:</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">printf</span> <span class="string">"<span class="variable">$(cat script.lua)</span>"</span> | sha1sum</span><br></pre></td></tr></table></figure>
<p>或者:</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">echo</span> -n <span class="string">"<span class="variable">$(cat script.lua)</span>"</span> | sha1sum</span><br></pre></td></tr></table></figure>
<p>获取了对应的 sha1 值,可以在编程时预先在代码中写入 sha1 值。</p>
<h1 id="实例"><a href="#实例" class="headerlink" title="实例"></a>实例</h1><h2 id="生产者生产速度控制"><a href="#生产者生产速度控制" class="headerlink" title="生产者生产速度控制"></a>生产者生产速度控制</h2><p>在生产者-消费者模型中,假若下游消费速度较慢,可以使用队列等方式暂存生产者生产的任务,起到削峰填谷的作用。</p>
<p>然而,在上游生产速度极快时,队列容量过大也会造成问题。</p>
<p>首先,可以判断队列现有容量,如果队列已经过长,则暂停生产。然而在并发较大的情况下,同一瞬间读取到的队列长度相同且小于最大长度,那么生产者仍会继续生产,给下游带来巨大的压力。</p>
<p>如果直接使用计数器呢?同样的,在同一个瞬间有多个请求一起请求生产,计数器递增,如果计数器值超过了限定的最大值,将计数器回退。如果后续各个请求出现异常,计数器将不会回退值,导致上游无法继续生产。</p>
<h3 id="带有超时效果的脚本"><a href="#带有超时效果的脚本" class="headerlink" title="带有超时效果的脚本"></a>带有超时效果的脚本</h3><p>计数器不自动回退的问题,可以通过超时进行处理。由于即要计数,又要拥有超时,Redis 原有的 Hash 结构不能满足需求,可以通过 ZSET 模拟超时以及完成计数。将时间戳作为分数即可,每个请求需要有唯一标示。</p>
<p>利用 Redis 命令的原子性,通过 Lua 脚本实现相关逻辑,控制速度。</p>
<figure class="highlight lua"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">--[[</span><br><span class="line">Usage:</span><br><span class="line">redis-cli eval "$(cat incrby_not_overby_zset.lua)" 1 zset_name max_value client_req_id client_req_timestamp [expire_second]</span><br><span class="line"></span><br><span class="line">or</span><br><span class="line"></span><br><span class="line">redis-cli evalsha xxx 1 zset_name max_value client_req_id [expire_second]</span><br><span class="line">--]]</span></span><br><span class="line"><span class="keyword">if</span> #ARGV >=<span class="number">4</span> <span class="keyword">then</span></span><br><span class="line"> redis.call(<span class="string">"ZREMRANGEBYSCORE"</span>, KEYS[<span class="number">1</span>], <span class="number">0</span>, <span class="built_in">tonumber</span>(ARGV[<span class="number">3</span>]) - <span class="built_in">tonumber</span>(ARGV[<span class="number">4</span>]))</span><br><span class="line"><span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">if</span> <span class="built_in">tonumber</span>(redis.call(<span class="string">"ZCARD"</span>, KEYS[<span class="number">1</span>])) >= <span class="built_in">tonumber</span>(ARGV[<span class="number">1</span>]) <span class="keyword">then</span></span><br><span class="line"> <span class="keyword">return</span> -<span class="built_in">tonumber</span>(ARGV[<span class="number">1</span>])</span><br><span class="line"><span class="keyword">end</span></span><br><span class="line"></span><br><span class="line">redis.call(<span class="string">"ZADD"</span>, KEYS[<span class="number">1</span>], ARGV[<span class="number">3</span>], ARGV[<span class="number">2</span>])</span><br><span class="line"></span><br><span class="line"><span class="keyword">return</span> <span class="built_in">tonumber</span>(redis.call(<span class="string">"ZCARD"</span>, KEYS[<span class="number">1</span>]))</span><br></pre></td></tr></table></figure>
<p>Mac上可以通过如下方式测试命令:</p>
<figure class="highlight autohotkey"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">redis-cli eval <span class="string">"$(cat incrby_not_over_by_zset.lua)"</span> <span class="number">1</span> ks1 <span class="number">10</span> <span class="escape">`u</span>uidgen<span class="escape">` </span><span class="escape">`d</span>ate +<span class="var_expand">%s<span class="escape">` </span>10</span></span><br></pre></td></tr></table></figure>
<p>即 10 秒内只能允许 10 次生产。</p>
<p>但是上述方法存在一个隐患,即各个生产者的时间戳并不一定同步,有可能出现超前或者靠后的情况,造成意外的过期问题。</p>
<p>此处有人会疑问,为何不通过 <code>redis.call("TIME")[1]</code> 获取时间戳,一旦使用了这个方法,在 3.2 以下版本的 Redis 中将无法继续执行任何写入操作(参见SO上的 <a href="https://stackoverflow.com/questions/40013598/why-redis-cannot-execute-non-deterministic-commands-in-lua-script" target="_blank" rel="external">Why redis cannot execute non deterministic commands in lua script
</a> 答案)。</p>
<blockquote>
<p>The scripts are replicated to slaves by sending the script over and running it on the slave, so the script needs to always produce the same results every time it’s run or the data on the slave will diverge from the data on the master.</p>
<p>You could try the new ‘scripts effects replication’ in the same link if you need perform non deterministic operations in a script.</p>
</blockquote>
<p>脚本中的操作尚不能像 MySQL 的 Binlog 一样采用混合模式传输,对于随机等操作的值直接传输值。</p>
<h3 id="根据队列长度进行控制"><a href="#根据队列长度进行控制" class="headerlink" title="根据队列长度进行控制"></a>根据队列长度进行控制</h3><p>直接将判断队列长度和 PUSH 操作合二为一:</p>
<figure class="highlight lua"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">--[[</span><br><span class="line">Usage:</span><br><span class="line">redis-cli eval "$(cat incrby_not_over_by_llen.lua)" 2 queue_name max_value msg [LPUSH]</span><br><span class="line"></span><br><span class="line">or</span><br><span class="line"></span><br><span class="line">redis-cli evalsha xxx 2 queue_name max_value msg [LPUSH]</span><br><span class="line">--]]</span></span><br><span class="line"><span class="keyword">local</span> PUSH = <span class="string">"LPUSH"</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">if</span> #ARGV >= <span class="number">3</span> <span class="keyword">then</span></span><br><span class="line"> PUSH = <span class="string">"RPUSH"</span></span><br><span class="line"><span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">if</span> redis.call(<span class="string">"LLEN"</span>, KEYS[<span class="number">1</span>]) >= <span class="built_in">tonumber</span>(ARGV[<span class="number">1</span>]) <span class="keyword">then</span></span><br><span class="line"> <span class="keyword">return</span> -<span class="built_in">tonumber</span>(ARGV[<span class="number">1</span>])</span><br><span class="line"><span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">return</span> redis.call(<span class="string">"LPUSH"</span>, KEYS[<span class="number">1</span>], ARGV[<span class="number">2</span>])</span><br></pre></td></tr></table></figure>
<p>这个方法的问题是每次都会带上需要加入的数据,如果生产者单次需要PUSH数据量小,并且获取数据的成本比较低,可以考虑直接如此改造。</p>
<h2 id="优先级队列"><a href="#优先级队列" class="headerlink" title="优先级队列"></a>优先级队列</h2><p>来看开源项目 <a href="https://github.com/redisson/redisson" target="_blank" rel="external">Redisson</a> 中 Lua 的应用:<a href="https://github.com/redisson/redisson/blob/master/redisson/src/main/java/org/redisson/RedissonPriorityQueue.java" target="_blank" rel="external">RedissonPriorityQueue</a>。</p>
<p>着重来看添加方法:</p>
<figure class="highlight cs"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br></pre></td><td class="code"><pre><span class="line">@<span class="function">Override</span><br><span class="line"><span class="keyword">public</span> boolean <span class="title">add</span>(<span class="params">V <span class="keyword">value</span></span>) </span>{</span><br><span class="line"> <span class="keyword">lock</span>.<span class="keyword">lock</span>();</span><br><span class="line"> </span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> checkComparator();</span><br><span class="line"> </span><br><span class="line"> BinarySearchResult<V> res = binarySearch(<span class="keyword">value</span>, codec);</span><br><span class="line"> <span class="keyword">int</span> index = <span class="number">0</span>;</span><br><span class="line"> <span class="keyword">if</span> (res.getIndex() < <span class="number">0</span>) {</span><br><span class="line"> index = -(res.getIndex() + <span class="number">1</span>);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> index = res.getIndex() + <span class="number">1</span>;</span><br><span class="line"> }</span><br><span class="line"> </span><br><span class="line"> commandExecutor.evalWrite(getName(), RedisCommands.EVAL_VOID, </span><br><span class="line"> <span class="string">"local len = redis.call('llen', KEYS[1]);"</span></span><br><span class="line"> + <span class="string">"if tonumber(ARGV[1]) < len then "</span></span><br><span class="line"> + <span class="string">"local pivot = redis.call('lindex', KEYS[1], ARGV[1]);"</span></span><br><span class="line"> + <span class="string">"redis.call('linsert', KEYS[1], 'before', pivot, ARGV[2]);"</span></span><br><span class="line"> + <span class="string">"return;"</span></span><br><span class="line"> + <span class="string">"end;"</span></span><br><span class="line"> + <span class="string">"redis.call('rpush', KEYS[1], ARGV[2]);"</span>, </span><br><span class="line"> Arrays.<Object>asList(getName()), </span><br><span class="line"> index, encode(<span class="keyword">value</span>));</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">true</span>;</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> <span class="keyword">lock</span>.unlock();</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<p>中间的 Lua 代码即通过二分查找,在队列中寻找新元素的位置,如果在队列中间则插入,否则则追加到队尾。</p>
<h1 id="参考"><a href="#参考" class="headerlink" title="参考"></a>参考</h1><ul>
<li><a href="http://redisbook.readthedocs.io/en/latest/feature/scripting.html" target="_blank" rel="external">http://redisbook.readthedocs.io/en/latest/feature/scripting.html</a></li>
<li><a href="http://redisdoc.com/script/eval.html" target="_blank" rel="external">http://redisdoc.com/script/eval.html</a></li>
<li><a href="https://github.com/hellokangning/Redis-note/blob/master/20.%20Lua%E8%84%9A%E6%9C%AC.md" target="_blank" rel="external">https://github.com/hellokangning/Redis-note/blob/master/20.%20Lua%E8%84%9A%E6%9C%AC.md</a></li>
<li><a href="http://wiki.jikexueyuan.com/project/redis/lua.html" target="_blank" rel="external">http://wiki.jikexueyuan.com/project/redis/lua.html</a></li>
</ul>
<h1 id="TL-DR"><a href="#TL-DR" class="headerlink" title="TL;DR"></a>TL;DR</h1><p>Redis 中使用 Lua 的相关笔记。</p>
<!-- lua-in-redis -->
亚庇
http://www.liaoaoyang.com/articles/2018/07/28/a-tour-to-kota-kinabalu/
2018-07-28T15:29:31.000Z
2018-07-30T16:05:59.590Z
<p>在亚庇的山水之间度过了一个慢节奏的周末。</p>
<!-- a-tour-to-kota-kinabalu -->
<a id="more"></a>
<p><img src="https://blog.wislay.com/wp-content/uploads/2018/07/IMG_4904.jpg" alt="DJI1"></p>
<p>(来自 DF 的 Mavic Pro)</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2018/07/IMG_4905.jpg" alt="DJI2"></p>
<p>(来自 DF 的 Mavic Pro)</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2018/07/end_of_island.jpg" alt="End of island"></p>
<p>长日将尽,下一个10000天会是如何呢?</p>
<p><img src="https://blog.wislay.com/wp-content/uploads/2018/07/sunset.jpg" alt="Sunset"></p>
<p>在亚庇的山水之间度过了一个慢节奏的周末。</p>
<!-- a-tour-to-kota-kinabalu -->
降低使用PHP Curl_multi_* 时的 Load
http://www.liaoaoyang.com/articles/2018/06/01/solution-of-high-load-when-using-php-multi-curl/
2018-06-01T01:51:09.000Z
2018-07-30T16:05:59.610Z
<h1 id="TL-DR"><a href="#TL-DR" class="headerlink" title="TL;DR"></a>TL;DR</h1><p>如果选择通过 <code>curl_multi_*</code> 函数并行发起请求,需要在使用 <code>curl_multi_select</code> 返回 -1 时增加休眠时间以降低 load。形如(代码来自 Guzzle):</p>
<figure class="highlight xquery"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">if</span> (<span class="variable">$this-</span>>active &&</span><br><span class="line"> curl_multi_select(<span class="variable">$this-</span>>_mh, <span class="variable">$this-</span>>selectTimeout) === -<span class="number">1</span></span><br><span class="line">) {</span><br><span class="line"> usleep(<span class="number">250</span>);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<p>使用的软件版本为:</p>
<ul>
<li>PHP 5.4.41</li>
<li>libcurl 7.19</li>
<li>Guzzle 5.3</li>
</ul>
<!-- solution-of-high-load-when-using-php-multi-curl -->
<a id="more"></a>
<h1 id="curlmulti"><a href="#curlmulti" class="headerlink" title="curlmulti*"></a>curl<em>multi</em>*</h1><p>如果要并发的发起请求,PHP 可以通过多进程发起请求的方式实现,每个请求由一个进程执行,然而这一方式开发以及运行成本较高,带来了繁重的进程管理以及 IPC 的编码工作。</p>
<p>进程是重量级的,而连接较之进程是轻量级的,结合IO多路复用可以让并发发起网络请求的成本降低,<code>curl_multi*</code> 这一组函数就是 PHP 基于 <code>libcurl</code> 封装的并发发起请求的工具。</p>
<p>网络上一些博文声称这一族函数是多线程发起请求,然而<a href="https://curl.haxx.se/libcurl/c/libcurl-tutorial.html" target="_blank" rel="external">libcurl官网</a>的文档中的 <strong>The multi Interface</strong> 部分说得很清楚,与这些博文相反:</p>
<blockquote>
<p>The multi interface, on the other hand, allows your program to transfer multiple files in both directions at the same time, without forcing you to use multiple threads. The name might make it seem that the multi interface is for multi-threaded programs, but the truth is almost the reverse. The multi interface allows a single-threaded application to perform the same kinds of multiple, simultaneous transfers that multi-threaded programs can perform.</p>
</blockquote>
<p>编码上,通过 <a href="http://php.net/manual/en/function.curl-multi-init.php" target="_blank" rel="external"><code>curl_multi_init</code></a> 创建并行请求处理对象,加入多个普通的 curl 对象;通过 <a href="http://php.net/manual/en/function.curl-multi-exec.php" target="_blank" rel="external"><code>curl_multi_exec</code></a> 按照内置的状态机建立连接,传输数据;使用 <a href="http://php.net/manual/en/function.curl-multi-select.php" target="_blank" rel="external"><code>curl_multi_select</code></a> 获取活跃的连接;之后尝试读取当中活跃连接的信息,记录对端返回的数据。</p>
<p>还是以 Guzzle 作为样板,选出最重要的执行过程,参见文件 <code>guzzlehttp/ringphp/src/Client/CurlMultiHandler.php</code>:</p>
<figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">/**</span><br><span class="line"> * Runs until all outstanding connections have completed.</span><br><span class="line"> */</span></span><br><span class="line"><span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">execute</span><span class="params">()</span></span><br><span class="line"></span>{</span><br><span class="line"> <span class="keyword">do</span> {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="variable">$this</span>->active &&</span><br><span class="line"> curl_multi_select(<span class="variable">$this</span>->_mh, <span class="variable">$this</span>->selectTimeout) === -<span class="number">1</span></span><br><span class="line"> ) {</span><br><span class="line"> <span class="comment">// Perform a usleep if a select returns -1.</span></span><br><span class="line"> <span class="comment">// See: https://bugs.php.net/bug.php?id=61141</span></span><br><span class="line"> usleep(<span class="number">250</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// Add any delayed futures if needed.</span></span><br><span class="line"> <span class="keyword">if</span> (<span class="variable">$this</span>->delays) {</span><br><span class="line"> <span class="variable">$this</span>->addDelays();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">do</span> {</span><br><span class="line"> <span class="variable">$mrc</span> = curl_multi_exec(<span class="variable">$this</span>->_mh, <span class="variable">$this</span>->active);</span><br><span class="line"> } <span class="keyword">while</span> (<span class="variable">$mrc</span> === CURLM_CALL_MULTI_PERFORM);</span><br><span class="line"></span><br><span class="line"> <span class="variable">$this</span>->processMessages();</span><br><span class="line"></span><br><span class="line"> <span class="comment">// If there are delays but no transfers, then sleep for a bit.</span></span><br><span class="line"> <span class="keyword">if</span> (!<span class="variable">$this</span>->active && <span class="variable">$this</span>->delays) {</span><br><span class="line"> usleep(<span class="number">500</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> } <span class="keyword">while</span> (<span class="variable">$this</span>->active || <span class="variable">$this</span>->handles);</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">private</span> <span class="function"><span class="keyword">function</span> <span class="title">processMessages</span><span class="params">()</span></span><br><span class="line"></span>{</span><br><span class="line"> <span class="keyword">while</span> (<span class="variable">$done</span> = curl_multi_info_read(<span class="variable">$this</span>->_mh)) {</span><br><span class="line"> <span class="variable">$id</span> = (int) <span class="variable">$done</span>[<span class="string">'handle'</span>];</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (!<span class="keyword">isset</span>(<span class="variable">$this</span>->handles[<span class="variable">$id</span>])) {</span><br><span class="line"> <span class="comment">// Probably was cancelled.</span></span><br><span class="line"> <span class="keyword">continue</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="variable">$entry</span> = <span class="variable">$this</span>->handles[<span class="variable">$id</span>];</span><br><span class="line"> <span class="variable">$entry</span>[<span class="string">'response'</span>][<span class="string">'transfer_stats'</span>] = curl_getinfo(<span class="variable">$done</span>[<span class="string">'handle'</span>]);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="variable">$done</span>[<span class="string">'result'</span>] !== CURLM_OK) {</span><br><span class="line"> <span class="variable">$entry</span>[<span class="string">'response'</span>][<span class="string">'curl'</span>][<span class="string">'errno'</span>] = <span class="variable">$done</span>[<span class="string">'result'</span>];</span><br><span class="line"> <span class="keyword">if</span> (function_exists(<span class="string">'curl_strerror'</span>)) {</span><br><span class="line"> <span class="variable">$entry</span>[<span class="string">'response'</span>][<span class="string">'curl'</span>][<span class="string">'error'</span>] = curl_strerror(<span class="variable">$done</span>[<span class="string">'result'</span>]);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="variable">$result</span> = CurlFactory::createResponse(</span><br><span class="line"> <span class="variable">$this</span>,</span><br><span class="line"> <span class="variable">$entry</span>[<span class="string">'request'</span>],</span><br><span class="line"> <span class="variable">$entry</span>[<span class="string">'response'</span>],</span><br><span class="line"> <span class="variable">$entry</span>[<span class="string">'headers'</span>],</span><br><span class="line"> <span class="variable">$entry</span>[<span class="string">'body'</span>]</span><br><span class="line"> );</span><br><span class="line"></span><br><span class="line"> <span class="variable">$this</span>->removeProcessed(<span class="variable">$id</span>);</span><br><span class="line"> <span class="variable">$entry</span>[<span class="string">'deferred'</span>]->resolve(<span class="variable">$result</span>);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<h1 id="curl-multi-select-之后的-usleep"><a href="#curl-multi-select-之后的-usleep" class="headerlink" title="curl_multi_select 之后的 usleep()"></a>curl_multi_select 之后的 usleep()</h1><p>Guzzle 作为优秀的 PHP HTTP 客户端,实现上在 <code>curl_multi_select</code> 返回 <strong>-1</strong> 时会休眠,这是降低 load 的一个看起来比较奇怪的关键操作。</p>
<h2 id="不带-sleep-usleep-的实现导致的问题"><a href="#不带-sleep-usleep-的实现导致的问题" class="headerlink" title="不带 sleep/usleep 的实现导致的问题"></a>不带 sleep/usleep 的实现导致的问题</h2><p>有部分的代码实现上并没有在这一情况下进行休眠,导致的问题是 CPU 使用率过高,导致服务器 load 上升。</p>
<p>load 与 CPU 使用时间有关系,当 CPU 繁忙时,服务器 load 会升高。常见的让服务器 CPU 飙升的操作就是死循环或者执行时间极短的不带休眠的循环。</p>
<p>例如以下的实现:</p>
<figure class="highlight xquery"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line"><span class="variable">$running</span> = <span class="literal">true</span>;</span><br><span class="line"><span class="keyword">while</span> (<span class="variable">$running</span>)</span><br><span class="line">{</span><br><span class="line"> if (CURLM_CALL_MULTI_PERFORM == curl_multi_exec(<span class="variable">$mh</span>, <span class="variable">$running</span>)) {</span><br><span class="line"> continue;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> curl_multi_select(<span class="variable">$mh</span>, <span class="variable">$selectTimeout</span>);</span><br><span class="line"> </span><br><span class="line"> while (<span class="variable">$ret</span> = curl_multi_info_read(<span class="variable">$mh</span>, <span class="variable">$id</span>)) {</span><br><span class="line"> // ETC </span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<p>就会在服务器上就会引起服务器 load 上升。</p>
<h2 id="PHP-curl-multi-select-实现"><a href="#PHP-curl-multi-select-实现" class="headerlink" title="PHP curl_multi_select 实现"></a>PHP curl_multi_select 实现</h2><p>不带休眠的实现会引起服务器 load 上升,要从 PHP <code>curl_multi_select</code> 的实现说起。</p>
<p>PHP 内核源码中自带了 curl 扩展的源码,在 <code>ext/curl/multi.c</code> 文件中可以看到这一函数的实现:</p>
<figure class="highlight handlebars"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br></pre></td><td class="code"><pre><span class="line"><span class="xml">/* </span><span class="expression">{{{ <span class="variable">proto</span> <span class="variable">int</span> <span class="variable">curl</span>_<span class="variable">multi</span>_<span class="variable">select</span>(<span class="variable">resource</span> <span class="variable">mh</span>[, <span class="variable">double</span> <span class="variable">timeout</span>])</span><br><span class="line"> <span class="variable">Get</span> <span class="variable">all</span> <span class="variable">the</span> <span class="variable">sockets</span> <span class="variable">associated</span> <span class="variable"><span class="keyword">with</span></span> <span class="variable">the</span> <span class="variable">cURL</span> <span class="variable">extension</span>, <span class="variable">which</span> <span class="variable">can</span> <span class="variable">then</span> <span class="variable">be</span> <span class="string">"selected"</span> */</span><br><span class="line"><span class="variable">PHP</span>_<span class="variable">FUNCTION</span>(<span class="variable">curl</span>_<span class="variable">multi</span>_<span class="variable">select</span>)</span><br><span class="line">{</span><br><span class="line"> <span class="variable">zval</span> *<span class="variable">z</span>_<span class="variable">mh</span>;</span><br><span class="line"> <span class="variable">php</span>_<span class="variable">curlm</span> *<span class="variable">mh</span>;</span><br><span class="line"> <span class="variable">fd</span>_<span class="variable">set</span> <span class="variable">readfds</span>;</span><br><span class="line"> <span class="variable">fd</span>_<span class="variable">set</span> <span class="variable">writefds</span>;</span><br><span class="line"> <span class="variable">fd</span>_<span class="variable">set</span> <span class="variable">exceptfds</span>;</span><br><span class="line"> <span class="variable">int</span> <span class="variable">maxfd</span>;</span><br><span class="line"> <span class="variable">double</span> <span class="variable">timeout</span> = 1<span class="variable">.</span>0;</span><br><span class="line"> <span class="variable">struct</span> <span class="variable">timeval</span> <span class="variable">to</span>;</span><br><span class="line"></span><br><span class="line"> <span class="variable"><span class="keyword">if</span></span> (<span class="variable">zend</span>_<span class="variable">parse</span>_<span class="variable">parameters</span>(<span class="variable">ZEND</span>_<span class="variable">NUM</span>_<span class="variable">ARGS</span>() <span class="variable">TSRMLS</span>_<span class="variable">CC</span>, <span class="string">"r|d"</span>, &<span class="variable">z</span>_<span class="variable">mh</span>, &<span class="variable">timeout</span>) == <span class="variable">FAILURE</span>) {</span><br><span class="line"> <span class="variable">return</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="variable">ZEND</span>_<span class="variable">FETCH</span>_<span class="variable">RESOURCE</span>(<span class="variable">mh</span>, <span class="variable">php</span>_<span class="variable">curlm</span> *, &<span class="variable">z</span>_<span class="variable">mh</span>, <span class="variable">-</span>1, <span class="variable">le</span>_<span class="variable">curl</span>_<span class="variable">multi</span>_<span class="variable">handle</span>_<span class="variable">name</span>, <span class="variable">le</span>_<span class="variable">curl</span>_<span class="variable">multi</span>_<span class="variable">handle</span>);</span><br><span class="line"></span><br><span class="line"> _<span class="variable">make</span>_<span class="variable">timeval</span>_<span class="variable">struct</span>(&<span class="variable">to</span>, <span class="variable">timeout</span>);</span><br><span class="line"> </span><br><span class="line"> <span class="variable">FD</span>_<span class="variable">ZERO</span>(&<span class="variable">readfds</span>);</span><br><span class="line"> <span class="variable">FD</span>_<span class="variable">ZERO</span>(&<span class="variable">writefds</span>);</span><br><span class="line"> <span class="variable">FD</span>_<span class="variable">ZERO</span>(&<span class="variable">exceptfds</span>);</span><br><span class="line"></span><br><span class="line"> <span class="variable">curl</span>_<span class="variable">multi</span>_<span class="variable">fdset</span>(<span class="variable">mh-</span>><span class="variable">multi</span>, &<span class="variable">readfds</span>, &<span class="variable">writefds</span>, &<span class="variable">exceptfds</span>, &<span class="variable">maxfd</span>);</span><br><span class="line"> <span class="variable"><span class="keyword">if</span></span> (<span class="variable">maxfd</span> == <span class="variable">-</span>1) {</span><br><span class="line"> <span class="variable">RETURN</span>_<span class="variable">LONG</span>(<span class="variable">-</span>1);</span><br><span class="line"> }</span><br><span class="line"> <span class="variable">RETURN</span>_<span class="variable">LONG</span>(<span class="variable">select</span>(<span class="variable">maxfd</span> + 1, &<span class="variable">readfds</span>, &<span class="variable">writefds</span>, &<span class="variable">exceptfds</span>, &<span class="variable">to</span>));</span><br><span class="line">}</span><br><span class="line">/* }}</span><span class="xml">} */</span></span><br></pre></td></tr></table></figure>
<p>文档上提及:</p>
<blockquote>
<p>On success, returns the number of descriptors contained in the descriptor sets. This may be 0 if there was no activity on any of the descriptors. On failure, this function will return -1 on a select failure (from the underlying select system call).</p>
</blockquote>
<p>即 -1 只在 <code>select</code> 系统调用失败时返回。 </p>
<p>由于请求是用户主动发起的,select 系统调用的监听集合需要通过某种方式获取,在这个编程模型下,<code>curl_multi_fdset</code> 就是产生监听集合的方法。</p>
<p>然而在实现中除去 PHP 函数的一些基本操作,可以看到返回 -1 的情况还会在 libcurl 的库函数 <code>curl_multi_fdset</code> 的 <code>maxfd</code> 变量被修改为 -1 后出现。</p>
<p><a href="https://curl.haxx.se/libcurl/c/curl_multi_fdset.html" target="_blank" rel="external">libcurl 的 curl_multi_fdset</a> 文档里提到的一段话值得注意:</p>
<blockquote>
<p>If no file descriptors are set by libcurl, max_fd will contain -1 when this function returns. Otherwise it will contain the highest descriptor number libcurl set. When libcurl returns -1 in max_fd, it is because libcurl currently does something that isn’t possible for your application to monitor with a socket and unfortunately you can then not know exactly when the current action is completed using select(). You then need to wait a while before you proceed and call curl_multi_perform anyway. How long to wait? Unless curl_multi_timeout gives you a lower number, we suggest 100 milliseconds or so, but you may want to test it out in your own particular conditions to find a suitable value.</p>
</blockquote>
<p>即 max_fd 返回 -1 时,需要主动休眠 100ms 或者根据实际情况决定。</p>
<p>在 PHP 执行过程中,我们无法判断是 select 系统调用返回的 -1 还是 <code>curl_multi_fdset</code> 的 <code>max_fd</code> 返回的 -1。</p>
<p>而当 <code>curl_multi_fdset</code> 的 <code>max_fd</code> 返回 -1 时,说明 fd 集合中没有可以读写的 fd,应当避免频繁轮询 <code>curl_multi_fdset</code> 这个函数,导致占满 CPU。</p>
<h1 id="结论"><a href="#结论" class="headerlink" title="结论"></a>结论</h1><p>推广到 PHP 扩展方法 <code>curl_multi_select</code> 的调用,可以得知当返回 <strong>-1</strong> 时,不应该继续轮询请求 <code>curl_mutli_exec</code> / <code>curl_multi_select</code>,应当主动休眠,降低CPU占用。</p>
<h1 id="TL-DR"><a href="#TL-DR" class="headerlink" title="TL;DR"></a>TL;DR</h1><p>如果选择通过 <code>curl_multi_*</code> 函数并行发起请求,需要在使用 <code>curl_multi_select</code> 返回 -1 时增加休眠时间以降低 load。形如(代码来自 Guzzle):</p>
<figure class="highlight xquery"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">if</span> (<span class="variable">$this-</span>>active &&</span><br><span class="line"> curl_multi_select(<span class="variable">$this-</span>>_mh, <span class="variable">$this-</span>>selectTimeout) === -<span class="number">1</span></span><br><span class="line">) {</span><br><span class="line"> usleep(<span class="number">250</span>);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<p>使用的软件版本为:</p>
<ul>
<li>PHP 5.4.41</li>
<li>libcurl 7.19</li>
<li>Guzzle 5.3</li>
</ul>
<!-- solution-of-high-load-when-using-php-multi-curl -->
PHP设计模式学习笔记
http://www.liaoaoyang.com/articles/2018/05/12/notes-of-php-design-patterns/
2018-05-12T15:22:31.000Z
2018-06-12T13:52:06.321Z
<h1 id="TL-DR"><a href="#TL-DR" class="headerlink" title="TL;DR"></a>TL;DR</h1><p>设计模式有助于写出组织结构更为合理的代码,少数实现上也和语言特性有所关系。</p>
<p>设计模式的样例以及说明网上都存在很多的样例,本文作为学习笔记,只简要记录学习过程中的个人理解的一些要点,如有错误,烦请指出。</p>
<!-- notes-of-php-design-patterns -->
<a id="more"></a>
<h1 id="笔记"><a href="#笔记" class="headerlink" title="笔记"></a>笔记</h1><h2 id="创建性模式"><a href="#创建性模式" class="headerlink" title="创建性模式"></a>创建性模式</h2><h3 id="单例模式"><a href="#单例模式" class="headerlink" title="单例模式"></a>单例模式</h3><ul>
<li>构造方法变为私有(不能通过<code>new</code>关键字产生对象)</li>
<li>只能通过实例获取方法获得实例</li>
<li>所有操作共享一个静态对象</li>
<li><code>__clone</code>魔术方法要被禁用防止对象被copy</li>
<li><code>__wake</code>魔术方法要禁用防止被反序列化</li>
</ul>
<h3 id="简单工厂模式"><a href="#简单工厂模式" class="headerlink" title="简单工厂模式"></a>简单工厂模式</h3><ul>
<li>实现一个带有工厂方法的类,负责生成其他类的对象</li>
<li>一次只能返回一个对象</li>
</ul>
<h3 id="工厂方法模式"><a href="#工厂方法模式" class="headerlink" title="工厂方法模式"></a>工厂方法模式</h3><ul>
<li>用于生成多个有共同特性的对象</li>
<li>多个工厂必须继承相同的接口或者抽象类</li>
<li>工厂生成的各类对象必须继承相同的接口或者抽象类</li>
</ul>
<h3 id="抽象工厂模式"><a href="#抽象工厂模式" class="headerlink" title="抽象工厂模式"></a>抽象工厂模式</h3><ul>
<li>工厂类包含多种方法,可以生成多个对象</li>
<li>在工厂方法中约束了可以产生的各个类型的对象</li>
<li>其他类似工厂方法模式</li>
</ul>
<h3 id="对象池模式"><a href="#对象池模式" class="headerlink" title="对象池模式"></a>对象池模式</h3><ul>
<li>管理多个单例</li>
</ul>
<h3 id="原型模式"><a href="#原型模式" class="headerlink" title="原型模式"></a>原型模式</h3><ul>
<li>用于复杂对象的创建</li>
<li>本质就是实现对象的深复制</li>
</ul>
<h2 id="结构性模式"><a href="#结构性模式" class="headerlink" title="结构性模式"></a>结构性模式</h2><h3 id="适配器模式"><a href="#适配器模式" class="headerlink" title="适配器模式"></a>适配器模式</h3><ul>
<li>对外定义统一的操作方法</li>
<li>根据操作对象的特性,在统一的操作方法中完成逻辑</li>
</ul>
<h3 id="桥接模式"><a href="#桥接模式" class="headerlink" title="桥接模式"></a>桥接模式</h3><ul>
<li>解决多个服务提供方对于同一类型的操作接口实现不同的问题</li>
<li>定义一个接口,用于统一对多个服务提供方同类操作</li>
<li>在桥接类中定义一个抽象操作方法,在具体的实现类中实现这一方法,完成实际业务代码和多个服务提供方的操作组合</li>
</ul>
<h3 id="组合模式"><a href="#组合模式" class="headerlink" title="组合模式"></a>组合模式</h3><ul>
<li>解决需要根据一定情况执行同类操作并且结果需要组合的问题</li>
<li>每个操作对象实现同样的操作接口,操作者逐一执行</li>
</ul>
<h3 id="装饰器模式"><a href="#装饰器模式" class="headerlink" title="装饰器模式"></a>装饰器模式</h3><ul>
<li>解决需要重复继承带来的方法重复臃肿问题</li>
<li>在实现共性方法之外,确定操作的各个阶段的接口</li>
<li>对于有各个属性的或者操作的类型,编写装饰器,实现各阶段操作的接口,对原始对象进行操作</li>
</ul>
<h3 id="依赖注入"><a href="#依赖注入" class="headerlink" title="依赖注入"></a>依赖注入</h3><ul>
<li>需要拆分类当中可以替换的部分</li>
<li>将操作需要的对象或者属性在方法参数中提供</li>
</ul>
<h3 id="门面模式"><a href="#门面模式" class="headerlink" title="门面模式"></a>门面模式</h3><ul>
<li>通过一个简单方法隐藏操作细节</li>
<li>实现一个方法,操作多个多个依赖对象和完成多个逻辑</li>
</ul>
<h3 id="链式操作"><a href="#链式操作" class="headerlink" title="链式操作"></a>链式操作</h3><ul>
<li>在每个允许的链式操作操作之后返回当前对象</li>
</ul>
<h3 id="代理模式"><a href="#代理模式" class="headerlink" title="代理模式"></a>代理模式</h3><ul>
<li>代理模式与适配器模式的区别在于代理模式下代理和第三方对象都继承与同一个接口</li>
<li>意义在于在代理中新增逻辑可以不用修改原始对象,反之亦然</li>
</ul>
<h3 id="注册器模式"><a href="#注册器模式" class="headerlink" title="注册器模式"></a>注册器模式</h3><ul>
<li>将后续可能使用到的对象记录到一个全局空间之中</li>
</ul>
<h2 id="行为性模式"><a href="#行为性模式" class="headerlink" title="行为性模式"></a>行为性模式</h2><h3 id="观察者模式"><a href="#观察者模式" class="headerlink" title="观察者模式"></a>观察者模式</h3><ul>
<li>解决一个操作需要触发多个关联操作的问题</li>
<li>原始对象需要实现注册和通知方法,记录所有注册的观察者</li>
<li>观察者要实现用于通知触发响应的接口,使用前注册</li>
</ul>
<h3 id="责任链模式"><a href="#责任链模式" class="headerlink" title="责任链模式"></a>责任链模式</h3><ul>
<li>对于同一类操作,实现多个处理者</li>
<li>对于一个处理者,声明无法处理时的下一个处理者,并在自身无法处理时,调用下一个处理者</li>
</ul>
<h3 id="模板方法"><a href="#模板方法" class="headerlink" title="模板方法"></a>模板方法</h3><ul>
<li>实现虚基类,提取出共性方法并实现逻辑,共性逻辑不得改变</li>
<li>对于共性方法需要依赖的操作对象声明为虚方法,要求子类实现</li>
</ul>
<h3 id="策略模式"><a href="#策略模式" class="headerlink" title="策略模式"></a>策略模式</h3><ul>
<li>解决多个判断条件下带来的面条式代码问题</li>
<li>基于控制反转(IoC),即将影响逻辑的方法从外部对象传入</li>
<li>声明一个逻辑操作接口,各个策略实现这一个接口</li>
<li>根据现有条件,确定使用策略并将策略类传递给逻辑操作的主体</li>
<li>逻辑操作的主体调用策略类的操作接口</li>
</ul>
<h3 id="访问者模式"><a href="#访问者模式" class="headerlink" title="访问者模式"></a>访问者模式</h3><ul>
<li>解决需要新增逻辑,但是又无法重写对象的问题</li>
<li>实现一个访问方法,根据注入到该方法的对象,访问该方法的某一个方法,实现逻辑的扩展</li>
</ul>
<h3 id="状态模式"><a href="#状态模式" class="headerlink" title="状态模式"></a>状态模式</h3><ul>
<li>与策略模式不同的是,不通过在外部确定策略,而是在策略类内部实现非当前策略的策略逻辑操作接口的触发</li>
<li>这个方式会加重策略类</li>
</ul>
<h3 id="命令模式"><a href="#命令模式" class="headerlink" title="命令模式"></a>命令模式</h3><ul>
<li>允许外部直接通过名称执行特定的逻辑</li>
</ul>
<h1 id="TL-DR"><a href="#TL-DR" class="headerlink" title="TL;DR"></a>TL;DR</h1><p>设计模式有助于写出组织结构更为合理的代码,少数实现上也和语言特性有所关系。</p>
<p>设计模式的样例以及说明网上都存在很多的样例,本文作为学习笔记,只简要记录学习过程中的个人理解的一些要点,如有错误,烦请指出。</p>
<!-- notes-of-php-design-patterns -->
Yaf集成Eloquent——使用事务以及DB Facade
http://www.liaoaoyang.com/articles/2018/05/03/integrate-yaf-with-eloquent-ii/
2018-05-02T16:41:50.000Z
2018-09-14T16:18:56.302Z
<h1 id="TL-DR"><a href="#TL-DR" class="headerlink" title="TL;DR"></a>TL;DR</h1><p>集成方法参见 <a href="/articles/2017/08/01/integrate-yaf-with-eloquent/">Yaf集成Eloquent</a> 。</p>
<p>集成基类的多个 Model 如果要正确的运行事务,需要保证各个 Model 的实例使用的是同一个数据库连接,在代码上可以通过共用同一个 <code>Illuminate\Database\Capsule\Manager</code> 对象实现。</p>
<p>使用 DB Facade 需要为 Facade 提供已经关联了 <code>db</code> 作为键,以 <code>Illuminate\Database\Capsule\Manager</code> 的实例为值的容器。</p>
<!-- integrate-yaf-with-eloquent-ii -->
<a id="more"></a>
<h1 id="实验环境"><a href="#实验环境" class="headerlink" title="实验环境"></a>实验环境</h1><ul>
<li>MySQL 5.6</li>
<li>PHP 5.6.31</li>
<li>Yaf 2.3.3</li>
<li>Eloquent 4.2.17(5.0亦可)</li>
</ul>
<h1 id="事务"><a href="#事务" class="headerlink" title="事务"></a>事务</h1><h2 id="Laravel-中的数据库事务"><a href="#Laravel-中的数据库事务" class="headerlink" title="Laravel 中的数据库事务"></a>Laravel 中的数据库事务</h2><p>Laravel 中的数据库事务可以通过 <a href="https://laravel.com/docs/5.0/database#database-transactions" target="_blank" rel="external">DB Facade</a> 开启:</p>
<figure class="highlight css"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="tag">DB</span><span class="pseudo">::transaction(function()</span></span><br><span class="line"><span class="rules">{</span><br><span class="line"> <span class="rule"><span class="attribute">DB</span>:<span class="value">:<span class="function">table</span>(<span class="string">'users'</span>)-><span class="function">update</span>([<span class="string">'votes'</span> => <span class="number">1</span>])</span></span>;</span><br><span class="line"></span><br><span class="line"> <span class="rule"><span class="attribute">DB</span>:<span class="value">:<span class="function">table</span>(<span class="string">'posts'</span>)-><span class="function">delete</span>()</span></span>;</span><br><span class="line">}</span>);</span><br></pre></td></tr></table></figure>
<p>即通过通过 DB Facade 访问了服务容器内的各个 Model 发起了事务。</p>
<h2 id="MySQL-事务"><a href="#MySQL-事务" class="headerlink" title="MySQL 事务"></a>MySQL 事务</h2><p>MySQL 中执行事务有两大前提条件:</p>
<ul>
<li>同一个数据库</li>
<li>同一个数据库连接</li>
</ul>
<p>如果使用 PDO 连接 MySQL 发起事务,那么 PDO 对象应该只有一个。</p>
<h2 id="原有样例存在的问题"><a href="#原有样例存在的问题" class="headerlink" title="原有样例存在的问题"></a>原有样例存在的问题</h2><p>部分单独使用 Eloquent 的样例,包括自己早期的尝试在内,都存在一个类似的情况:</p>
<figure class="highlight xml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br></pre></td><td class="code"><pre><span class="line"><span class="php"><span class="preprocessor"><?php</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">use</span> <span class="title">Illuminate</span>\<span class="title">Database</span>\<span class="title">Capsule</span>\<span class="title">Manager</span> <span class="title">as</span> <span class="title">IlluminateCapsule</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">Illuminate</span>\<span class="title">Database</span>\<span class="title">Eloquent</span>\<span class="title">Model</span> <span class="title">as</span> <span class="title">IlluminateModel</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">Yaf</span>\<span class="title">Registry</span> <span class="title">as</span> <span class="title">YRegistry</span>;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">BaseModel</span> <span class="keyword">extends</span> <span class="title">IlluminateModel</span></span><br><span class="line"></span>{</span><br><span class="line"> <span class="keyword">protected</span> <span class="variable">$config</span> = <span class="keyword">null</span>;</span><br><span class="line"> <span class="keyword">protected</span> <span class="variable">$capsule</span> = <span class="keyword">null</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">__construct</span><span class="params">(array <span class="variable">$attributes</span> = array<span class="params">()</span>)</span></span><br><span class="line"> </span>{</span><br><span class="line"> <span class="keyword">parent</span>::__construct(<span class="variable">$attributes</span>);</span><br><span class="line"> <span class="variable">$dbConfigKey</span> = DATABASE_CONFIG_KEY;</span><br><span class="line"> <span class="variable">$this</span>->config = YRegistry::get(<span class="string">'config'</span>);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (!<span class="variable">$this</span>->config-><span class="variable">$dbConfigKey</span>) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> <span class="keyword">Exception</span>(<span class="string">"Must configure database in .ini first"</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="variable">$this</span>->config = <span class="variable">$this</span>->config-><span class="variable">$dbConfigKey</span>->toArray();</span><br><span class="line"> <span class="variable">$this</span>->capsule = <span class="keyword">new</span> IlluminateCapsule();</span><br><span class="line"> <span class="variable">$this</span>->capsule->addConnection(<span class="variable">$this</span>->config);</span><br><span class="line"> <span class="variable">$this</span>->capsule->bootEloquent();</span><br><span class="line"> }</span><br><span class="line">}</span></span><br></pre></td></tr></table></figure>
<p>这里在构造函数中会构建一个 <code>Illuminate\Database\Capsule\Manager $capsue</code> 对象,这一对象是 Eloquent 的关键部分之一,实现了连接管理等功能:</p>
<figure class="highlight xml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br></pre></td><td class="code"><pre><span class="line"><span class="php"><span class="preprocessor"><?php</span> <span class="keyword">namespace</span> <span class="title">Illuminate</span>\<span class="title">Database</span>\<span class="title">Capsule</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">use</span> <span class="title">PDO</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">Illuminate</span>\<span class="title">Events</span>\<span class="title">Dispatcher</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">Illuminate</span>\<span class="title">Cache</span>\<span class="title">CacheManager</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">Illuminate</span>\<span class="title">Container</span>\<span class="title">Container</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">Illuminate</span>\<span class="title">Database</span>\<span class="title">DatabaseManager</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">Illuminate</span>\<span class="title">Database</span>\<span class="title">Eloquent</span>\<span class="title">Model</span> <span class="title">as</span> <span class="title">Eloquent</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">Illuminate</span>\<span class="title">Database</span>\<span class="title">Connectors</span>\<span class="title">ConnectionFactory</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">Illuminate</span>\<span class="title">Support</span>\<span class="title">Traits</span>\<span class="title">CapsuleManagerTrait</span>;</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">Manager</span> </span>{</span><br><span class="line"></span><br><span class="line"> <span class="keyword">use</span> <span class="title">CapsuleManagerTrait</span>;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span><br><span class="line"> * The database manager instance.</span><br><span class="line"> *</span><br><span class="line"> * <span class="doctag">@var</span> \Illuminate\Database\DatabaseManager</span><br><span class="line"> */</span></span><br><span class="line"> <span class="keyword">protected</span> <span class="variable">$manager</span>;</span></span><br></pre></td></tr></table></figure>
<p>通过内置的 <code>protected \Illuminate\Database\DatabaseManager $manager</code> 对象,建立真正的连接。</p>
<p>Manager 会根据声明的连接配置名称创建对应的连接,通过连接配置名称可以直接获取对应的连接对象,注入到实际执行查询操作的 <code>Illuminate\Database\Query\Builder</code> 中完成查询。</p>
<p>连接是按序连接,在需要进行查询时,<code>\Illuminate\Database\DatabaseManager</code> 会调用 <code>\Illuminate\Database\Connectors\ConnectionFactory</code> 成员的 <code>make()</code> 方法创建连接,在这一方法中完成连接对象的创建和连接操作。</p>
<figure class="highlight xquery"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br></pre></td><td class="code"><pre><span class="line"><?php <span class="keyword">namespace</span> Illuminate\Database\Connectors;</span><br><span class="line"></span><br><span class="line">use PDO;</span><br><span class="line">use Illuminate\Container\Container;</span><br><span class="line">use Illuminate\Database\MySqlConnection;</span><br><span class="line">use Illuminate\Database\SQLiteConnection;</span><br><span class="line">use Illuminate\Database\PostgresConnection;</span><br><span class="line">use Illuminate\Database\SqlServerConnection;</span><br><span class="line"></span><br><span class="line">class ConnectionFactory {</span><br><span class="line"></span><br><span class="line"> /**</span><br><span class="line"> * The IoC container instance.</span><br><span class="line"> *</span><br><span class="line"> * @var \Illuminate\Container\Container</span><br><span class="line"> */</span><br><span class="line"> protected <span class="variable">$container</span>;</span><br><span class="line"></span><br><span class="line"> /**</span><br><span class="line"> * Create a new connection factory instance.</span><br><span class="line"> *</span><br><span class="line"> * @param \Illuminate\Container\Container <span class="variable">$container</span></span><br><span class="line"> * @return void</span><br><span class="line"> */</span><br><span class="line"> public function __construct(Container <span class="variable">$container</span>)</span><br><span class="line"> {</span><br><span class="line"> <span class="variable">$this-</span>>container = <span class="variable">$container</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> /**</span><br><span class="line"> * Establish a PDO connection based on the configuration.</span><br><span class="line"> *</span><br><span class="line"> * @param array <span class="variable">$config</span></span><br><span class="line"> * @param string <span class="variable">$name</span></span><br><span class="line"> * @return \Illuminate\Database\Connection</span><br><span class="line"> */</span><br><span class="line"> public function make(array <span class="variable">$config</span>, <span class="variable">$name</span> = null)</span><br><span class="line"> {</span><br><span class="line"> <span class="variable">$config</span> = <span class="variable">$this-</span>>parseConfig(<span class="variable">$config</span>, <span class="variable">$name</span>);</span><br><span class="line"></span><br><span class="line"> if (isset(<span class="variable">$config</span>[<span class="string">'read'</span>]))</span><br><span class="line"> {</span><br><span class="line"> return <span class="variable">$this-</span>>createReadWriteConnection(<span class="variable">$config</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> return <span class="variable">$this-</span>>createSingleConnection(<span class="variable">$config</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> /**</span><br><span class="line"> * Create a single database connection instance.</span><br><span class="line"> *</span><br><span class="line"> * @param array <span class="variable">$config</span></span><br><span class="line"> * @return \Illuminate\Database\Connection</span><br><span class="line"> */</span><br><span class="line"> protected function createSingleConnection(array <span class="variable">$config</span>)</span><br><span class="line"> {</span><br><span class="line"> <span class="variable">$pdo</span> = <span class="variable">$this-</span>>createConnector(<span class="variable">$config</span>)->connect(<span class="variable">$config</span>);</span><br><span class="line"></span><br><span class="line"> return <span class="variable">$this-</span>>createConnection(<span class="variable">$config</span>[<span class="string">'driver'</span>], <span class="variable">$pdo</span>, <span class="variable">$config</span>[<span class="string">'database'</span>], <span class="variable">$config</span>[<span class="string">'prefix'</span>], <span class="variable">$config</span>);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
<p>那么问题来了,在多个 Model 实现类进行事务操作时,对于每一个子类来说,基类 <code>BaseModel</code> 中都会建立一个全新的 <code>DatabaseManager</code> 对象,实际上对于每个 Model 来说都建立了不同的数据库连接,不同连接下执行事务是不可能保证 ACID 的。</p>
<h2 id="解决"><a href="#解决" class="headerlink" title="解决"></a>解决</h2><h3 id="Laravel-中的处理"><a href="#Laravel-中的处理" class="headerlink" title="Laravel 中的处理"></a>Laravel 中的处理</h3><p>来看原生支持 Eloquent 的 Laravel 是如何处理的。</p>
<p>阅读 <code>Illuminate\Database\DatabaseServiceProvider</code> 的源码:</p>
<figure class="highlight xquery"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br></pre></td><td class="code"><pre><span class="line">/**</span><br><span class="line"> * Register the primary database bindings.</span><br><span class="line"> *</span><br><span class="line"> * @<span class="keyword">return</span> void</span><br><span class="line"> */</span><br><span class="line">protected <span class="keyword">function</span> registerConnectionServices()</span><br><span class="line">{</span><br><span class="line"> // The connection factory is used to create the actual connection instances on</span><br><span class="line"> // the database. We will inject the factory into the manager so that it may</span><br><span class="line"> // make the connections while they are actually needed and not of before.</span><br><span class="line"> <span class="variable">$this-</span>>app->singleton(<span class="string">'db.factory'</span>, function (<span class="variable">$app</span>) {</span><br><span class="line"> return new ConnectionFactory(<span class="variable">$app</span>);</span><br><span class="line"> });</span><br><span class="line"></span><br><span class="line"> // The database manager is used to resolve various connections, since multiple</span><br><span class="line"> // connections might be managed. It also implements the connection resolver</span><br><span class="line"> // interface which may be used by other components requiring connections.</span><br><span class="line"> <span class="variable">$this-</span>>app->singleton(<span class="string">'db'</span>, function (<span class="variable">$app</span>) {</span><br><span class="line"> return new DatabaseManager(<span class="variable">$app</span>, <span class="variable">$app</span>[<span class="string">'db.factory'</span>]);</span><br><span class="line"> });</span><br><span class="line"></span><br><span class="line"> <span class="variable">$this-</span>>app->bind(<span class="string">'db.connection'</span>, function (<span class="variable">$app</span>) {</span><br><span class="line"> return <span class="variable">$app</span>[<span class="string">'db'</span>]->connection();</span><br><span class="line"> });</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<p>Laravel的处理很简单,将 <code>\Illuminate\Database\DatabaseManager</code> 处理成单例。</p>
<h3 id="本文解决方案"><a href="#本文解决方案" class="headerlink" title="本文解决方案"></a>本文解决方案</h3><p><a href="https://github.com/liaoaoyang/YafWithEloquentSample/blob/master/application/models/Base.php" target="_blank" rel="external">样例</a> 中使用了同样的方式,即将此对象实现为单例模式。</p>
<h1 id="DB-Facade"><a href="#DB-Facade" class="headerlink" title="DB Facade"></a>DB Facade</h1><p>通过 Facade 可以方便的访问数据库相关功能。</p>
<p>Eloquent 中 <code>Illuminate\Support\Facades\DB</code> 即是入口。</p>
<h2 id="Laravel-中的-DB-Facade"><a href="#Laravel-中的-DB-Facade" class="headerlink" title="Laravel 中的 DB Facade"></a>Laravel 中的 DB Facade</h2><p>Laravel 中的 Facade 实际上是通过访问已关联的应用程序对象中已关联的对象,通过对象实现的 <strong>call 以及 </strong>callStatic 魔术方法,实现功能。</p>
<p>摘录部分 DB Facade 代码:</p>
<figure class="highlight xquery"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br></pre></td><td class="code"><pre><span class="line">protected static <span class="keyword">function</span> getFacadeAccessor()</span><br><span class="line">{</span><br><span class="line"> return <span class="string">'db'</span>;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">/**</span><br><span class="line"> * Resolve the facade root <span class="keyword">instance</span> from the container.</span><br><span class="line"> *</span><br><span class="line"> * @param string|object <span class="variable">$name</span></span><br><span class="line"> * @<span class="keyword">return</span> mixed</span><br><span class="line"> */</span><br><span class="line">protected static <span class="keyword">function</span> resolveFacadeInstance(<span class="variable">$name</span>)</span><br><span class="line">{</span><br><span class="line"> if (is_object(<span class="variable">$name</span>)) {</span><br><span class="line"> return <span class="variable">$name</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> if (isset(static::<span class="variable">$resolvedInstance</span>[<span class="variable">$name</span>])) {</span><br><span class="line"> return static::<span class="variable">$resolvedInstance</span>[<span class="variable">$name</span>];</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> return static::<span class="variable">$resolvedInstance</span>[<span class="variable">$name</span>] = static::<span class="variable">$app</span>[<span class="variable">$name</span>];</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">/**</span><br><span class="line"> * Get the root object behind the facade.</span><br><span class="line"> *</span><br><span class="line"> * @<span class="keyword">return</span> mixed</span><br><span class="line"> */</span><br><span class="line">public static <span class="keyword">function</span> getFacadeRoot()</span><br><span class="line">{</span><br><span class="line"> return static::resolveFacadeInstance(static::getFacadeAccessor());</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">/**</span><br><span class="line"> * Handle dynamic, static calls <span class="keyword">to</span> the object.</span><br><span class="line"> *</span><br><span class="line"> * @param string <span class="variable">$method</span></span><br><span class="line"> * @param <span class="keyword">array</span> <span class="variable">$args</span></span><br><span class="line"> * @<span class="keyword">return</span> mixed</span><br><span class="line"> *</span><br><span class="line"> * @throws \RuntimeException</span><br><span class="line"> */</span><br><span class="line">public static <span class="keyword">function</span> __callStatic(<span class="variable">$method</span>, <span class="variable">$args</span>)</span><br><span class="line">{</span><br><span class="line"> <span class="variable">$instance</span> = static::getFacadeRoot();</span><br><span class="line"></span><br><span class="line"> if (! <span class="variable">$instance</span>) {</span><br><span class="line"> throw new RuntimeException(<span class="string">'A facade root has not been set.'</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> return <span class="variable">$instance-</span>><span class="variable">$method</span>(...<span class="variable">$args</span>);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<p>可以看到,DB Facade 访问的是 <code>$app['db']</code> 对应的对象。</p>
<p>回到之前 <code>Illuminate\Database\DatabaseServiceProvider</code> 的 <code>registerConnectionServices</code> 方法:</p>
<figure class="highlight xquery"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="variable">$this-</span>>app->singleton(<span class="string">'db'</span>, <span class="keyword">function</span> (<span class="variable">$app</span>) {</span><br><span class="line"> return new DatabaseManager(<span class="variable">$app</span>, <span class="variable">$app</span>[<span class="string">'db.factory'</span>]);</span><br><span class="line">});</span><br></pre></td></tr></table></figure>
<p>实际上 <code>$app['db']</code> 关联的是一个 <code>DatabaseManager</code> 对象。</p>
<h2 id="文中的-DB-Facade"><a href="#文中的-DB-Facade" class="headerlink" title="文中的 DB Facade"></a>文中的 DB Facade</h2><p>按照这一思路,将 <code>Illuminate\Support\Facades\DB</code> 的 $app 对象增加一个 key db,并关联上当前存在的 <code>DatabaseManager</code> 对象即可:</p>
<figure class="highlight xquery"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">self::<span class="variable">$capsule</span> = new IlluminateCapsule();</span><br><span class="line">self::<span class="variable">$capsule-</span>>bootEloquent();</span><br><span class="line">Illuminate\Support\Facades\DB::setFacadeApplication([</span><br><span class="line"> <span class="string">'db'</span> => self::<span class="variable">$capsule-</span>>getDatabaseManager(),</span><br><span class="line">]);</span><br></pre></td></tr></table></figure>
<h1 id="TL-DR"><a href="#TL-DR" class="headerlink" title="TL;DR"></a>TL;DR</h1><p>集成方法参见 <a href="/articles/2017/08/01/integrate-yaf-with-eloquent/">Yaf集成Eloquent</a> 。</p>
<p>集成基类的多个 Model 如果要正确的运行事务,需要保证各个 Model 的实例使用的是同一个数据库连接,在代码上可以通过共用同一个 <code>Illuminate\Database\Capsule\Manager</code> 对象实现。</p>
<p>使用 DB Facade 需要为 Facade 提供已经关联了 <code>db</code> 作为键,以 <code>Illuminate\Database\Capsule\Manager</code> 的实例为值的容器。</p>
<!-- integrate-yaf-with-eloquent-ii -->
PHP中Redis/MySQL的长连接
http://www.liaoaoyang.com/articles/2018/04/30/persistent-connections-of-redis-and-pdo-extension-in-php/
2018-04-30T09:29:47.000Z
2018-04-30T16:42:09.552Z
<h1 id="TL-DR"><a href="#TL-DR" class="headerlink" title="TL;DR"></a>TL;DR</h1><p>PHP 中针对 Redis / MySQL 的长连接是生命周期级别的长连接,对于同一个进程的每一次相同目标的请求都不会释放当前连接对象。而针对 TCP Socket 级别的连接是否已断开,则交给操作系统维持。</p>
<p>使用 PDO 对 MySQL 开启持久连接,要注意 PHP 执行的进程数量,不能超过 MySQL 设定的最大连接数。</p>
<p>上述结论的前提是使用 phpredis 扩展,PHP 版本为 5.4.41。</p>
<a id="more"></a>
<h1 id="背景"><a href="#背景" class="headerlink" title="背景"></a>背景</h1><p>假设某同学使用 PHP 开发了一个队列消费 daemon,某天业务压力较大。实现上每次与 Redis 通信都新建连接,同时本地端口范围过小,导致用尽了本地端口,无法建立连接。</p>
<p>这个时候使用 pconnect 可以有效的减少重复建立连接的成本。使用 <code>ss</code> 等工具可以看到相关的连接数目。</p>
<p>编写如下脚本进行测试:</p>
<figure class="highlight xquery"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br></pre></td><td class="code"><pre><span class="line"><?php</span><br><span class="line"> <span class="variable">$persist</span> = isset(<span class="variable">$argv</span>[<span class="number">1</span>]) ? <span class="literal">true</span> : <span class="literal">false</span>;</span><br><span class="line"> <span class="variable">$cmd</span> = <span class="string">"ss -ant | grep ESTAB | awk 'BEGIN{conn=0;}{if($5 == \"</span><span class="number">127.0</span>.<span class="number">0</span>.<span class="number">1</span>:<span class="number">6379</span>\<span class="string">"){++conn;}}END{print conn}'"</span>;</span><br><span class="line"></span><br><span class="line"> echo <span class="string">"Before:\n"</span>;</span><br><span class="line"> echo shell_exec(<span class="variable">$cmd</span>) . <span class="string">"\n"</span>;</span><br><span class="line"></span><br><span class="line"> <span class="variable">$rs</span> = [];</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> (<span class="variable">$i</span> = <span class="number">1</span>; <span class="variable">$i</span> <= <span class="number">5</span>; ++<span class="variable">$i</span>) {</span><br><span class="line"> <span class="variable">$r</span> = new Redis();</span><br><span class="line"> <span class="variable">$rs</span>[] = <span class="variable">$r</span>;</span><br><span class="line"></span><br><span class="line"> if (<span class="variable">$persist</span>) {</span><br><span class="line"> <span class="variable">$r-</span>>pconnect(<span class="string">'127.0.0.1'</span>, <span class="number">6379</span>);</span><br><span class="line"> } else {</span><br><span class="line"> <span class="variable">$r-</span>>connect(<span class="string">'127.0.0.1'</span>, <span class="number">6379</span>);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> echo <span class="string">"After:\n"</span>;</span><br><span class="line"> echo shell_exec(<span class="variable">$cmd</span>) . <span class="string">"\n"</span>;</span><br></pre></td></tr></table></figure>
<p>结果:</p>
<figure class="highlight avrasm"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line">root@vm:~<span class="preprocessor"># php test_redis_connect.php persist</span></span><br><span class="line"><span class="label">Before:</span></span><br><span class="line"><span class="number">0</span></span><br><span class="line"></span><br><span class="line"><span class="label">After:</span></span><br><span class="line"><span class="number">1</span></span><br><span class="line"></span><br><span class="line">root@vm:~<span class="preprocessor"># php test_redis_connect.php</span></span><br><span class="line"><span class="label">Before:</span></span><br><span class="line"><span class="number">0</span></span><br><span class="line"></span><br><span class="line"><span class="label">After:</span></span><br><span class="line"><span class="number">5</span></span><br></pre></td></tr></table></figure>
<p>可以看到使用了 pconnect 可以有效减少连接数。</p>
<h2 id="Redis-连接的实现"><a href="#Redis-连接的实现" class="headerlink" title="Redis 连接的实现"></a>Redis 连接的实现</h2><p>直接翻看 phpredis 扩展源码(2.2.7)。</p>
<p>总体而言,phpredis 通过给定的参数产生一个 <code>RedisSock*</code>,在这一个结构中包含一个 <code>php_stream</code> 结构,利用这一个流式成员,完成与服务端之间的网络通信。</p>
<p>连接时使用 <code>pconnect</code> 与 <code>connect</code> 的区别在于对于流式对象的管理方式。</p>
<h3 id="pconnect-实现"><a href="#pconnect-实现" class="headerlink" title="pconnect 实现"></a>pconnect 实现</h3><h4 id="参数传递"><a href="#参数传递" class="headerlink" title="参数传递"></a>参数传递</h4><p>扩展中连接方法的入口是 <code>redis.c</code> 文件中的 <code>redis_connect</code> 函数。解析参数列表部分的逻辑为:</p>
<figure class="highlight scala"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">if</span> (zend_parse_method_parameters(<span class="type">ZEND_NUM_ARGS</span>() <span class="type">TSRMLS_CC</span>, getThis(), <span class="string">"Os|ldsl"</span>,</span><br><span class="line"> &<span class="class"><span class="keyword">object</span>, <span class="title">redis_ce</span>, <span class="title">&host</span>, <span class="title">&host_len</span>, <span class="title">&port</span>,</span><br><span class="line"></span> &timeout, &persistent_id, &persistent_id_len,</span><br><span class="line"> &retry_interval) == <span class="type">FAILURE</span>) {</span><br><span class="line"> <span class="keyword">return</span> <span class="type">FAILURE</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<p>可以看到有一个可选的字符串参数(<code>|</code>之后的<code>s</code>)<code>persistent_id</code>,此处先留意这一个变量,后续会使用到。同时,也可以看到 <code>pconnect</code> 方法是可以在调用时指定这个参数的值的。</p>
<p>在后续的 <code>redis_sock_create</code> 函数中,会将当前的 <code>persistent_id</code> 赋值给返回 <code>RedisSock</code> 对象,以供后续创建连接使用。</p>
<h4 id="连接"><a href="#连接" class="headerlink" title="连接"></a>连接</h4><p>建立与 Redis 服务器之间的连接工作实际上是由 <code>redis_sock_server_open</code> 函数完成的,即在这一个方法中,通过 <code>php_stream_xport_create</code> 函数将网络连接创建并赋值给 <code>RedisSock</code> 中的 <code>stream</code> 变量,实际上,后续的所有网络读写操作,都会通过这一个变量完成。</p>
<figure class="highlight lasso"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">if</span> (redis_sock<span class="subst">-></span>persistent) {</span><br><span class="line"> <span class="keyword">if</span> (redis_sock<span class="subst">-></span>persistent_id) {</span><br><span class="line"> spprintf(<span class="subst">&</span>persistent_id, <span class="number">0</span>, <span class="string">"phpredis:%s:%s"</span>, host, redis_sock<span class="subst">-></span>persistent_id);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> spprintf(<span class="subst">&</span>persistent_id, <span class="number">0</span>, <span class="string">"phpredis:%s:%f"</span>, host, redis_sock<span class="subst">-></span>timeout);</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">redis_sock<span class="subst">-></span>stream = php_stream_xport_create(host, host_len, ENFORCE_SAFE_MODE,</span><br><span class="line"> STREAM_XPORT_CLIENT</span><br><span class="line"> | STREAM_XPORT_CONNECT,</span><br><span class="line"> persistent_id, tv_ptr, <span class="built_in">NULL</span>, <span class="subst">&</span>errstr, <span class="subst">&</span>err</span><br><span class="line"> );</span><br></pre></td></tr></table></figure>
<p>这里判断 <code>persistent</code> 的逻辑,就用到了传递进来的 <code>persistent_id</code> 参数,如果没有设定,会根据 Redis 服务器的 IP 和当前设定的超时创建出一个字符串作为值。</p>
<p>这个 ID 用于标记这是一个需要保持的对象(持久性资源)。</p>
<h4 id="PHP-持久性资源"><a href="#PHP-持久性资源" class="headerlink" title="PHP 持久性资源"></a>PHP 持久性资源</h4><p>首先,对于Socket/文件等对象,在 PHP 中都是资源对象,而 PHP 扩展中在实现上会将这一个资源对象记录到哈希表 <code>EG(regular_list)</code> 中,通过 <code>zend_list_delete</code> 等函数完成资源引用计数的操作,当引用计数为 0 时,认为资源已无效,删除资源。</p>
<p>在 <code>redis.c</code> 中的 <code>redis_connect</code> 方法中,当通过 redis_sock_create 创建成功资源对象后,通过 <code>zend_list_insert</code> 完成了当前资源对象的记录:</p>
<figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// redis.c</span></span><br><span class="line"> redis_sock = redis_sock_create(host, host_len, port, timeout, persistent, persistent_id, retry_interval, <span class="number">0</span>);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (redis_sock_server_open(redis_sock, <span class="number">1</span> TSRMLS_CC) < <span class="number">0</span>) {</span><br><span class="line"> redis_free_socket(redis_sock);</span><br><span class="line"> <span class="keyword">return</span> FAILURE;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"><span class="preprocessor">#<span class="keyword">if</span> PHP_VERSION_ID >= <span class="number">50400</span></span></span><br><span class="line"> id = zend_list_insert(redis_sock, le_redis_sock TSRMLS_CC);</span><br><span class="line"><span class="preprocessor">#<span class="keyword">else</span></span></span><br><span class="line"> id = zend_list_insert(redis_sock, le_redis_sock);</span><br><span class="line"><span class="preprocessor">#<span class="keyword">endif</span></span></span><br></pre></td></tr></table></figure>
<figure class="highlight axapta"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// zend_list.c</span></span><br><span class="line">ZEND_API <span class="keyword">int</span> zend_list_insert(<span class="keyword">void</span> *ptr, <span class="keyword">int</span> type TSRMLS_DC)</span><br><span class="line">{</span><br><span class="line"> <span class="keyword">int</span> <span class="keyword">index</span>;</span><br><span class="line"> zend_rsrc_list_entry le;</span><br><span class="line"></span><br><span class="line"> le.ptr=ptr;</span><br><span class="line"> le.type=type;</span><br><span class="line"> le.refcount=<span class="number">1</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">index</span> = zend_hash_next_free_element(&EG(regular_list));</span><br><span class="line"></span><br><span class="line"> zend_hash_index_update(&EG(regular_list), <span class="keyword">index</span>, (<span class="keyword">void</span> *) &le, sizeof(zend_rsrc_list_entry), NULL);</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">index</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<p>回到 phpredis 扩展连接创建过程,在创建过程中的 <code>_php_stream_xport_create</code> 函数中,当设定了 <code>persistent_id</code> 之后,会首先在另一个全局哈希表 <code>EG(persistent_list)</code> 中通过 <code>persistent_id</code> 查找到对应的资源对象是否存在,如果存在还会将其尝试注册到 <code>EG(regular_list)</code> 中,减少了无谓的创建过程。</p>
<figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br></pre></td><td class="code"><pre><span class="line"><span class="function">PHPAPI <span class="keyword">int</span> <span class="title">php_stream_from_persistent_id</span><span class="params">(<span class="keyword">const</span> <span class="keyword">char</span> *persistent_id, php_stream **stream TSRMLS_DC)</span></span><br><span class="line"></span>{</span><br><span class="line"> zend_rsrc_list_entry *le;</span><br><span class="line"></span><br><span class="line"><span class="comment">// 注意</span></span><br><span class="line"> <span class="keyword">if</span> (zend_hash_find(&EG(persistent_list), (<span class="keyword">char</span>*)persistent_id, <span class="built_in">strlen</span>(persistent_id)+<span class="number">1</span>, (<span class="keyword">void</span>*) &le) == SUCCESS) {</span><br><span class="line"> <span class="keyword">if</span> (Z_TYPE_P(le) == le_pstream) {</span><br><span class="line"> <span class="keyword">if</span> (stream) {</span><br><span class="line"> HashPosition pos;</span><br><span class="line"> zend_rsrc_list_entry *regentry;</span><br><span class="line"> ulong index = -<span class="number">1</span>; <span class="comment">/* intentional */</span></span><br><span class="line"></span><br><span class="line"> <span class="comment">/* see if this persistent resource already has been loaded to the</span><br><span class="line"> * regular list; allowing the same resource in several entries in the</span><br><span class="line"> * regular list causes trouble (see bug #54623) */</span></span><br><span class="line"> zend_hash_internal_pointer_reset_ex(&EG(regular_list), &pos);</span><br><span class="line"> <span class="keyword">while</span> (zend_hash_get_current_data_ex(&EG(regular_list),</span><br><span class="line"> (<span class="keyword">void</span> **)&regentry, &pos) == SUCCESS) {</span><br><span class="line"> <span class="keyword">if</span> (regentry->ptr == le->ptr) {</span><br><span class="line"> zend_hash_get_current_key_ex(&EG(regular_list), <span class="literal">NULL</span>, <span class="literal">NULL</span>,</span><br><span class="line"> &index, <span class="number">0</span>, &pos);</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> zend_hash_move_forward_ex(&EG(regular_list), &pos);</span><br><span class="line"> }</span><br><span class="line"> </span><br><span class="line"> *stream = (php_stream*)le->ptr;</span><br><span class="line"> <span class="keyword">if</span> (index == -<span class="number">1</span>) { <span class="comment">/* not found in regular list */</span></span><br><span class="line"> le->refcount++;</span><br><span class="line"> (*stream)->rsrc_id = ZEND_REGISTER_RESOURCE(<span class="literal">NULL</span>, *stream, le_pstream);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> regentry->refcount++;</span><br><span class="line"> (*stream)->rsrc_id = index;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> PHP_STREAM_PERSISTENT_SUCCESS;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> PHP_STREAM_PERSISTENT_FAILURE;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> PHP_STREAM_PERSISTENT_NOT_EXIST;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<p>随后会检查连接的可用性,由于对 Redis 服务器是一个 TCP 连接,会通过 <code>xp_socket.c</code> 文件中的 <code>php_sockop_set_option</code> 中 <code>PHP_STREAM_OPTION_CHECK_LIVENESS</code> 对应分支的逻辑进行连接可用性的检查:</p>
<figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">static</span> <span class="keyword">int</span> <span class="title">php_sockop_set_option</span><span class="params">(php_stream *stream, <span class="keyword">int</span> option, <span class="keyword">int</span> value, <span class="keyword">void</span> *ptrparam TSRMLS_DC)</span></span><br><span class="line"></span>{</span><br><span class="line"> <span class="keyword">int</span> oldmode, flags;</span><br><span class="line"> <span class="keyword">php_netstream_data_t</span> *sock = (<span class="keyword">php_netstream_data_t</span>*)stream->abstract;</span><br><span class="line"> php_stream_xport_param *xparam;</span><br><span class="line"> </span><br><span class="line"> <span class="keyword">switch</span>(option) {</span><br><span class="line"> <span class="keyword">case</span> PHP_STREAM_OPTION_CHECK_LIVENESS:</span><br><span class="line"> {</span><br><span class="line"> <span class="keyword">struct</span> timeval tv;</span><br><span class="line"> <span class="keyword">char</span> buf;</span><br><span class="line"> <span class="keyword">int</span> alive = <span class="number">1</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (value == -<span class="number">1</span>) {</span><br><span class="line"> <span class="keyword">if</span> (sock->timeout.tv_sec == -<span class="number">1</span>) {</span><br><span class="line"> tv.tv_sec = FG(default_socket_timeout);</span><br><span class="line"> tv.tv_usec = <span class="number">0</span>;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> tv = sock->timeout;</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> tv.tv_sec = value;</span><br><span class="line"> tv.tv_usec = <span class="number">0</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (sock->socket == -<span class="number">1</span>) {</span><br><span class="line"> alive = <span class="number">0</span>;</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (php_pollfd_for(sock->socket, PHP_POLLREADABLE|POLLPRI, &tv) > <span class="number">0</span>) {</span><br><span class="line"> <span class="keyword">if</span> (<span class="number">0</span> >= recv(sock->socket, &buf, <span class="keyword">sizeof</span>(buf), MSG_PEEK) && php_socket_errno() != EWOULDBLOCK) {</span><br><span class="line"> alive = <span class="number">0</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> alive ? PHP_STREAM_OPTION_RETURN_OK : PHP_STREAM_OPTION_RETURN_ERR;</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">//...</span></span><br></pre></td></tr></table></figure>
<p>实际上是通过 <code>poll</code> 系统调用判断 socket 对应的 fd 是否可读并且 <code>recv</code> 系统调用返回可读长度小于等于0实现的。</p>
<p>对于 <code>EG(persistent_list)</code> 这一个哈希表,为什么能够实现针对于PHP应用程序持久性的连接呢?答案很简单,这个哈希表只有在 <strong>MSHUTDOWN</strong> 阶段时才会清理:</p>
<figure class="highlight handlebars"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line"><span class="xml">// zend.c</span><br><span class="line">void zend_shutdown(TSRMLS_D) /* </span><span class="expression">{{{ */</span><br><span class="line">{</span><br><span class="line"><span class="begin-block">#ifdef ZEND</span>_<span class="variable">SIGNALS</span></span><br><span class="line"> <span class="variable">zend</span>_<span class="variable">signal</span>_<span class="variable">shutdown</span>(<span class="variable">TSRMLS</span>_<span class="variable">C</span>);</span><br><span class="line"><span class="begin-block">#endif</span></span><br><span class="line"><span class="begin-block">#ifdef ZEND</span>_<span class="variable">WIN</span>32</span><br><span class="line"> <span class="variable">zend</span>_<span class="variable">shutdown</span>_<span class="variable">timeout</span>_<span class="variable">thread</span>();</span><br><span class="line"><span class="begin-block">#endif</span></span><br><span class="line"> <span class="variable">zend</span>_<span class="variable">destroy</span>_<span class="variable">rsrc</span>_<span class="variable">list</span>(&<span class="variable">EG</span>(<span class="variable">persistent</span>_<span class="variable">list</span>) <span class="variable">TSRMLS</span>_<span class="variable">CC</span>);</span><br><span class="line"> /<span class="end-block">/ ...</span></span></span><br></pre></td></tr></table></figure>
<p>而 <code>EG(regular_list)</code> 在 <strong>RSHUTDOWN</strong> 阶段就会被清理(这一阶段进程尚未退出):</p>
<figure class="highlight handlebars"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br></pre></td><td class="code"><pre><span class="line"><span class="xml">// zend.c</span><br><span class="line">void zend_deactivate(TSRMLS_D) /* </span><span class="expression">{{{ */</span><br><span class="line">{</span><br><span class="line"> /* <span class="variable">we</span>'<span class="variable">re</span> <span class="variable">no</span> <span class="variable">longer</span> <span class="variable">executing</span> <span class="variable">anything</span> */</span><br><span class="line"> <span class="variable">EG</span>(<span class="variable">opline</span>_<span class="variable">ptr</span>) = <span class="variable">NULL</span>;</span><br><span class="line"> <span class="variable">EG</span>(<span class="variable">active</span>_<span class="variable">symbol</span>_<span class="variable">table</span>) = <span class="variable">NULL</span>;</span><br><span class="line"></span><br><span class="line"> <span class="variable">zend</span>_<span class="variable">try</span> {</span><br><span class="line"> <span class="variable">shutdown</span>_<span class="variable">scanner</span>(<span class="variable">TSRMLS</span>_<span class="variable">C</span>);</span><br><span class="line"> } <span class="variable">zend</span>_<span class="variable">end</span>_<span class="variable">try</span>();</span><br><span class="line"></span><br><span class="line"> /* <span class="variable">shutdown</span>_<span class="variable">executor</span>() <span class="variable">takes</span> <span class="variable">care</span> <span class="variable">of</span> <span class="variable">its</span> <span class="variable">own</span> <span class="variable">bailout</span> <span class="variable">handling</span> */</span><br><span class="line"> <span class="variable">shutdown</span>_<span class="variable">executor</span>(<span class="variable">TSRMLS</span>_<span class="variable">C</span>);</span><br><span class="line"></span><br><span class="line"> <span class="variable">zend</span>_<span class="variable">try</span> {</span><br><span class="line"> <span class="variable">shutdown</span>_<span class="variable">compiler</span>(<span class="variable">TSRMLS</span>_<span class="variable">C</span>);</span><br><span class="line"> } <span class="variable">zend</span>_<span class="variable">end</span>_<span class="variable">try</span>();</span><br><span class="line"></span><br><span class="line"> <span class="variable">zend</span>_<span class="variable">destroy</span>_<span class="variable">rsrc</span>_<span class="variable">list</span>(&<span class="variable">EG</span>(<span class="variable">regular</span>_<span class="variable">list</span>) <span class="variable">TSRMLS</span>_<span class="variable">CC</span>);</span><br><span class="line"> /<span class="end-block">/ ...</span></span></span><br></pre></td></tr></table></figure>
<p>哪怕是显式的调用了 <code>close</code> 方法,也只是向服务器发送 <code>QUIT</code> 指令并将当前对象的状态设定为已断开状态,实际上并未清理对应的流:</p>
<figure class="highlight zephir"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// redis.c</span></span><br><span class="line">PHP_REDIS_API <span class="keyword">int</span> redis_sock_disconnect(RedisSock *redis_sock TSRMLS_DC)</span><br><span class="line">{</span><br><span class="line"> <span class="keyword">if</span> (redis_sock == <span class="keyword">NULL</span>) {</span><br><span class="line"> <span class="keyword">return</span> <span class="number">1</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> redis_sock->dbNumber = <span class="number">0</span>;</span><br><span class="line"> <span class="keyword">if</span> (redis_sock->stream != <span class="keyword">NULL</span>) {</span><br><span class="line"> <span class="keyword">if</span> (!redis_sock->persistent) {</span><br><span class="line"> redis_sock_write(redis_sock, <span class="string">"QUIT"</span> _NL, sizeof(<span class="string">"QUIT"</span> _NL) - <span class="number">1</span> TSRMLS_CC);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> redis_sock->status = REDIS_SOCK_STATUS_DISCONNECTED;</span><br><span class="line"> redis_sock->watching = <span class="number">0</span>;</span><br><span class="line"> <span class="keyword">if</span>(redis_sock->stream && !redis_sock->persistent) { <span class="comment">/* still valid after the write? */</span></span><br><span class="line"> php_stream_close(redis_sock->stream);</span><br><span class="line"> }</span><br><span class="line"> redis_sock->stream = <span class="keyword">NULL</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> <span class="number">1</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<h2 id="MySQL-PDO的连接实现"><a href="#MySQL-PDO的连接实现" class="headerlink" title="MySQL PDO的连接实现"></a>MySQL PDO的连接实现</h2><p>MySQL PDO的连接实现与 Redis 类似,也是在传递了持久连接的参数后,会将连接对象保存到持久性资源对象的哈希表中。</p>
<p>构建持久化标记ID的方式看起来比 phpredis 扩展稍好一些,用到了 <code>服务器Host/服务器端口/用户名/密码</code> 等元素。</p>
<figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// pdo_dbh.c</span></span><br><span class="line"><span class="comment">// ...</span></span><br><span class="line"> <span class="keyword">if</span> (SUCCESS == zend_hash_index_find(Z_ARRVAL_P(options), PDO_ATTR_PERSISTENT, (<span class="keyword">void</span>**)&v)) {</span><br><span class="line"> <span class="keyword">if</span> (Z_TYPE_PP(v) == IS_STRING && !is_numeric_string(Z_STRVAL_PP(v), Z_STRLEN_PP(v), <span class="literal">NULL</span>, <span class="literal">NULL</span>, <span class="number">0</span>) && Z_STRLEN_PP(v) > <span class="number">0</span>) {</span><br><span class="line"> <span class="comment">/* user specified key */</span></span><br><span class="line"> plen = spprintf(&hashkey, <span class="number">0</span>, <span class="string">"PDO:DBH:DSN=%s:%s:%s:%s"</span>, data_source,</span><br><span class="line"> username ? username : <span class="string">""</span>,</span><br><span class="line"> password ? password : <span class="string">""</span>,</span><br><span class="line"> Z_STRVAL_PP(v));</span><br><span class="line"> is_persistent = <span class="number">1</span>;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> convert_to_long_ex(v);</span><br><span class="line"> is_persistent = Z_LVAL_PP(v) ? <span class="number">1</span> : <span class="number">0</span>;</span><br><span class="line"> plen = spprintf(&hashkey, <span class="number">0</span>, <span class="string">"PDO:DBH:DSN=%s:%s:%s"</span>, data_source,</span><br><span class="line"> username ? username : <span class="string">""</span>,</span><br><span class="line"> password ? password : <span class="string">""</span>);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"><span class="comment">// ...</span></span><br></pre></td></tr></table></figure>
<p>然而如果启用了持久连接,PDO并没有给出主动清理 <code>EG(persistent_list)</code> 的方法,所以,如果要使用这一个特性,需要格外注意不要启动过多的进程,以至于超过 MySQL 设定的最大连接数。</p>
<h1 id="小结"><a href="#小结" class="headerlink" title="小结"></a>小结</h1><p>PHP 中如果使用了 Redis/MySQL 的持久连接功能,PHP 内核会通过将资源对象存储到 <strong>MSHUTDOWN</strong> 阶段才会清理的全局 HashTable 中,实现针对同一个服务器进行请求时,在 PHP 执行的整个生命周期之内保证了正常情况下不会重新创建连接。</p>
<h1 id="相关"><a href="#相关" class="headerlink" title="相关"></a>相关</h1><ul>
<li><a href="https://github.com/walu/phpbook/blob/master/7.1.md" target="_blank" rel="external">PHPBOOK-7.1 函数的参数</a></li>
<li><a href="https://github.com/walu/phpbook/blob/master/9.2.md" target="_blank" rel="external">PHPBOOK-9.2 PHP中的资源类型</a></li>
<li><a href="https://github.com/walu/phpbook/blob/master/14.1.md" target="_blank" rel="external">PHPBOOK-14.1 14.1 流的概览</a></li>
<li><a href="http://man7.org/linux/man-pages/man2/poll.2.html" target="_blank" rel="external">Linux Programmer’s Manual-POLL(2)</a></li>
</ul>
<h1 id="TL-DR"><a href="#TL-DR" class="headerlink" title="TL;DR"></a>TL;DR</h1><p>PHP 中针对 Redis / MySQL 的长连接是生命周期级别的长连接,对于同一个进程的每一次相同目标的请求都不会释放当前连接对象。而针对 TCP Socket 级别的连接是否已断开,则交给操作系统维持。</p>
<p>使用 PDO 对 MySQL 开启持久连接,要注意 PHP 执行的进程数量,不能超过 MySQL 设定的最大连接数。</p>
<p>上述结论的前提是使用 phpredis 扩展,PHP 版本为 5.4.41。</p>
在PHP-FPM中使用pcntl扩展
http://www.liaoaoyang.com/articles/2018/03/31/can-we-use-pcntl-in-fpm-fcgi-sapi/
2018-03-31T14:09:25.000Z
2018-04-12T15:25:36.000Z
<h1 id="TL-DR"><a href="#TL-DR" class="headerlink" title="TL;DR"></a>TL;DR</h1><p>在 PHP-FPM 中使用 pcntl 扩展在一定情况下是可行的,但是并不是合理的。</p>
<a id="more"></a>
<h1 id="pcntl-扩展"><a href="#pcntl-扩展" class="headerlink" title="pcntl 扩展"></a>pcntl 扩展</h1><p><a href="http://php.net/manual/zh/intro.pcntl.php" target="_blank" rel="external">pcntl</a> 扩展主要用于Unix方式的进程创建, 程序执行, 信号处理以及进程的中断。</p>
<p>手册上告知:</p>
<blockquote>
<p>Process Control should not be enabled within a web server environment and unexpected results may happen if any Process Control functions are used within a web server environment.</p>
</blockquote>
<p>但是,这个扩展真的不能在 LNMP 环境中使用吗?让我们来验证一下。</p>
<h1 id="多进程-fork-验证"><a href="#多进程-fork-验证" class="headerlink" title="多进程 fork 验证"></a>多进程 fork 验证</h1><p>pcntl 最常用的功能之一应该就是 fork 新进程了。通过 <code>pcntl_fork()</code> 函数可以使用 PHP 进行<a href="https://www.liaoaoyang.cn/articles/2017/08/22/php-daemon/" target="_blank" rel="external">多进程</a>应用程序的开发工作。</p>
<h2 id="环境"><a href="#环境" class="headerlink" title="环境"></a>环境</h2><p>还是使用 Docker 进行环境的准备工作。使用 OpenResty + PHP 5.6.35。</p>
<p>默认的 <code>php:5.6.35-fpm-alpine3.4</code> 镜像并没有开启 <code>pcntl</code> 扩展,所以需要自行修改镜像,Dockerfile 如下:</p>
<figure class="highlight livescript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">FROM <span class="attribute">php</span>:<span class="number">5.6</span>.<span class="number">35</span>-fpm-alpine3.<span class="number">4</span></span><br><span class="line">RUN apk add --<span class="literal">no</span>-cache --virtual .build-deps g++ make autoconf <span class="string">\</span></span><br><span class="line"> && docker-php-source extract <span class="string">\</span></span><br><span class="line"> && cd /usr/src/php <span class="string">\</span></span><br><span class="line"> && docker-php-ext-configure pcntl <span class="string">\</span></span><br><span class="line"> && docker-php-ext-install pcntl <span class="string">\</span></span><br><span class="line"> && docker-php-ext-enable pcntl <span class="string">\</span></span><br><span class="line"> && apk del --purge .build-deps</span><br></pre></td></tr></table></figure>
<p>构建镜像 <code>php:5.6.35-fpm-pcntl-alpine3.4</code>。</p>
<p>使用 docker-compose 编排 OpenResty 和 PHP-FPM,<code>docker-compose.yml</code>如下:</p>
<figure class="highlight groovy"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br></pre></td><td class="code"><pre><span class="line"><span class="string">version:</span> <span class="string">'3'</span></span><br><span class="line"><span class="string">networks:</span></span><br><span class="line"><span class="label"> pcntl_test:</span></span><br><span class="line"><span class="string">services:</span></span><br><span class="line"><span class="label"> pcntl_test_phpfpm:</span></span><br><span class="line"><span class="label"> image:</span> <span class="string">php:</span><span class="number">5.6</span><span class="number">.35</span>-fpm-pcntl-alpine3<span class="number">.4</span></span><br><span class="line"><span class="label"> networks:</span></span><br><span class="line"> - pcntl_test</span><br><span class="line"><span class="label"> volumes:</span></span><br><span class="line"> - <span class="regexp">/data/</span>www<span class="regexp">/htdocs/</span><span class="string">pcntl_test:</span><span class="regexp">/var/</span>www/html</span><br><span class="line"> - <span class="regexp">/data/</span>www<span class="regexp">/logs:/</span>var<span class="regexp">/www/</span>logs</span><br><span class="line"><span class="label"> expose:</span></span><br><span class="line"> - <span class="string">"9000"</span></span><br><span class="line"><span class="label"></span><br><span class="line"> pcntl_test_openresty:</span></span><br><span class="line"><span class="label"> image:</span> openresty/<span class="string">openresty:</span>alpine</span><br><span class="line"><span class="label"> networks:</span></span><br><span class="line"> - pcntl_test</span><br><span class="line"><span class="label"> depends_on:</span></span><br><span class="line"> - pcntl_test_phpfpm</span><br><span class="line"><span class="label"> links:</span></span><br><span class="line"> - pcntl_test_phpfpm</span><br><span class="line"><span class="label"> ports:</span></span><br><span class="line"> - <span class="string">"9527:80"</span></span><br><span class="line"><span class="label"> volumes:</span></span><br><span class="line"> - <span class="regexp">/data/</span>www<span class="regexp">/etc/</span>pcntl_test<span class="regexp">/conf/</span><span class="string">nginx:</span><span class="regexp">/etc/</span>nginx<span class="regexp">/conf.d/</span>:ro</span><br><span class="line"> - <span class="regexp">/data/</span>www<span class="regexp">/logs:/</span>usr<span class="regexp">/local/</span>openresty<span class="regexp">/nginx/</span>logs</span><br><span class="line"> - <span class="regexp">/data/</span>www<span class="regexp">/htdocs/</span><span class="string">pcntl_test:</span><span class="regexp">/var/</span>www/html</span><br></pre></td></tr></table></figure>
<h2 id="测试代码"><a href="#测试代码" class="headerlink" title="测试代码"></a>测试代码</h2><p>编写一个最简单的 fork 样例:</p>
<figure class="highlight xquery"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line"><?php</span><br><span class="line"> <span class="keyword">if</span> (extension_loaded(<span class="string">'pcntl'</span>)) {</span><br><span class="line"> echo <span class="string">"true\n"</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> (<span class="variable">$i</span> = <span class="number">0</span>; <span class="variable">$i</span> < <span class="number">5</span>; ++<span class="variable">$i</span>) {</span><br><span class="line"> <span class="variable">$pid</span> = pcntl_fork();</span><br><span class="line"></span><br><span class="line"> if (<span class="number">0</span> == <span class="variable">$pid</span>) {</span><br><span class="line"> echo <span class="string">"child:"</span> . posix_getpid() . <span class="string">"\n"</span>;</span><br><span class="line"> exit();</span><br><span class="line"> } else if (<span class="variable">$pid</span> > <span class="number">0</span>) {</span><br><span class="line"> echo <span class="string">"parent:"</span> . posix_getpid() . <span class="string">" {$pid}\n"</span>;</span><br><span class="line"> } else {</span><br><span class="line"> echo posix_getpid() . <span class="string">" error\n"</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
<h2 id="表现"><a href="#表现" class="headerlink" title="表现"></a>表现</h2><p>测试开始。</p>
<figure class="highlight groovy"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">curl <span class="string">http:</span><span class="comment">//127.0.0.1:9527/index.php</span></span><br></pre></td></tr></table></figure>
<p>预期的结果是首先会输出 true,毕竟已经加载了 pcntl 扩展,之后会输出各 5 行带有 <code>parent</code> 和 <code>child</code> 的字符串。形如:</p>
<figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line"><span class="literal">true</span></span><br><span class="line">parent:<span class="number">6</span> <span class="number">17</span></span><br><span class="line">parent:<span class="number">6</span> <span class="number">18</span></span><br><span class="line">parent:<span class="number">6</span> <span class="number">19</span></span><br><span class="line">parent:<span class="number">6</span> <span class="number">20</span></span><br><span class="line">parent:<span class="number">6</span> <span class="number">21</span></span><br><span class="line">child:<span class="number">21</span></span><br><span class="line">child:<span class="number">20</span></span><br><span class="line">child:<span class="number">18</span></span><br><span class="line">child:<span class="number">19</span></span><br><span class="line">child:<span class="number">17</span></span><br></pre></td></tr></table></figure>
<p>实际情况是:</p>
<figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="literal">true</span></span><br><span class="line">parent:<span class="number">6</span> <span class="number">17</span></span><br><span class="line">parent:<span class="number">6</span> <span class="number">18</span></span><br><span class="line">parent:<span class="number">6</span> <span class="number">19</span></span><br><span class="line">parent:<span class="number">6</span> <span class="number">20</span></span><br><span class="line">parent:<span class="number">6</span> <span class="number">21</span></span><br></pre></td></tr></table></figure>
<p>推测和现实产生了差异,乍一看似乎真的不能 fork 子进程,然而真实的执行情况是什么呢?</p>
<h3 id="echo-更换为写入文件"><a href="#echo-更换为写入文件" class="headerlink" title="echo 更换为写入文件"></a>echo 更换为写入文件</h3><p>我们将 echo 操作换成写入文件:</p>
<figure class="highlight xquery"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line"><?php</span><br><span class="line"> <span class="variable">$logFn</span> = dirname(__FILE_<span class="number">_</span>) . <span class="string">'/log'</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> (<span class="variable">$i</span> = <span class="number">0</span>; <span class="variable">$i</span> < <span class="number">5</span>; ++<span class="variable">$i</span>) {</span><br><span class="line"> <span class="variable">$pid</span> = pcntl_fork();</span><br><span class="line"></span><br><span class="line"> if (<span class="number">0</span> == <span class="variable">$pid</span>) {</span><br><span class="line"> file_put_contents(<span class="variable">$logFn</span>, <span class="string">"child:"</span> . posix_getpid() . <span class="string">"\n"</span>, FILE_APPEND);</span><br><span class="line"> exit();</span><br><span class="line"> } else if (<span class="variable">$pid</span> > <span class="number">0</span>) {</span><br><span class="line"> file_put_contents(<span class="variable">$logFn</span>, <span class="string">"parent:"</span> . posix_getpid() . <span class="string">" {$pid}\n"</span>, FILE_APPEND);</span><br><span class="line"> } else {</span><br><span class="line"> file_put_contents(<span class="variable">$logFn</span>, posix_getpid() . <span class="string">" error\n"</span>, FILE_APPEND);</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
<p>在结果文件中,出现了预期的结果。</p>
<p>那么,可以得出第一个结论:<code>pcntl</code> 在一定情况下是可以在 PHP-FPM 中使用的。</p>
<h3 id="子进程-echo-“未能执行”原因"><a href="#子进程-echo-“未能执行”原因" class="headerlink" title="子进程 echo “未能执行”原因"></a>子进程 echo “未能执行”原因</h3><p>PHP 使用的是缓冲 IO,存在输入输出缓冲区,很有可能在 PHP-FPM 被 WebServer 断开连接时,还没能刷新各自进程的输入输出缓冲区,导致结果无法输出。</p>
<p>我们在最初代码的基础之上再作一些改动,增加缓冲区刷新操作:</p>
<figure class="highlight xquery"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br></pre></td><td class="code"><pre><span class="line"><?php</span><br><span class="line"> <span class="keyword">for</span> (<span class="variable">$i</span> = <span class="number">0</span>; <span class="variable">$i</span> < <span class="number">5</span>; ++<span class="variable">$i</span>) {</span><br><span class="line"> <span class="variable">$pid</span> = pcntl_fork();</span><br><span class="line"></span><br><span class="line"> if (<span class="number">0</span> == <span class="variable">$pid</span>) {</span><br><span class="line"> echo <span class="string">"child:"</span> . posix_getpid() . <span class="string">"\n"</span>;</span><br><span class="line"> ob_flush();</span><br><span class="line"> flush();</span><br><span class="line"> exit();</span><br><span class="line"> } else if (<span class="variable">$pid</span> > <span class="number">0</span>) {</span><br><span class="line"> echo <span class="string">"parent:"</span> . posix_getpid() . <span class="string">" {$pid}\n"</span>;</span><br><span class="line"> ob_flush();</span><br><span class="line"> flush();</span><br><span class="line"> } else {</span><br><span class="line"> echo posix_getpid() . <span class="string">" error\n"</span>;</span><br><span class="line"> ob_flush();</span><br><span class="line"> flush();</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
<p>如此之后在操作系统上才有可能输出子进程的文本。</p>
<h1 id="为什么在-PHP-FPM-中使用-pcntl-是不合理的"><a href="#为什么在-PHP-FPM-中使用-pcntl-是不合理的" class="headerlink" title="为什么在 PHP-FPM 中使用 pcntl 是不合理的"></a>为什么在 PHP-FPM 中使用 pcntl 是不合理的</h1><h2 id="案例"><a href="#案例" class="headerlink" title="案例"></a>案例</h2><p>同样是刚才的第一段代码,场景可能是有人期望在 HTTP 请求中使用多进程提升处理能力,编写出类似这样的代码。</p>
<p>我们看看执行前后容器的进程数有何变化:</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">➜ docker top pcntltest_pcntl_<span class="built_in">test</span>_phpfpm_1 | grep php-fpm | wc <span class="operator">-l</span></span><br><span class="line"><span class="number">3</span></span><br><span class="line">➜ curl <span class="operator">-s</span> http://<span class="number">127.0</span>.<span class="number">0.1</span>:<span class="number">9527</span>/index.php > /dev/null</span><br><span class="line">➜ docker top pcntltest_pcntl_<span class="built_in">test</span>_phpfpm_1 | grep php-fpm | wc <span class="operator">-l</span></span><br><span class="line"><span class="number">8</span></span><br></pre></td></tr></table></figure>
<p>可以看到进程数增加了5个。这里有人可能会问,不是都已经 exit() 了吗?为什么进程数还增加了呢?</p>
<h2 id="PHP-生命周期与-PHP-FPM"><a href="#PHP-生命周期与-PHP-FPM" class="headerlink" title="PHP 生命周期与 PHP-FPM"></a>PHP 生命周期与 PHP-FPM</h2><p>PHP 的<a href="http://www.php-internals.com/book/?p=chapt02/02-01-php-life-cycle-and-zend-engine" target="_blank" rel="external">生命周期</a> TIPI 上已经说明得很清楚了,可以看到调用 <code>exit()</code> 后会发生的情况:</p>
<blockquote>
<p>请求处理完后就进入了结束阶段,一般脚本执行到末尾或者通过调用exit()或die()函数, PHP都将进入结束阶段。和开始阶段对应,结束阶段也分为两个环节,一个在请求结束后停用模块(RSHUTDOWN,对应RINIT), 一个在SAPI生命周期结束(Web服务器退出或者命令行脚本执行完毕退出)时关闭模块(MSHUTDOWN,对应MINIT)。</p>
</blockquote>
<p>也就是说,<code>exit()</code> 只是到达了执行的结束阶段,并不是退出进程,退出进程的时机由 Zend 引擎决定。</p>
<p>PHP-FPM 为了提高处理能力,fork 出进程执行 PHP 代码后并不会立刻退出,进程仍会运行一定时间才会退出。</p>
<p>综上,在 PHP-FPM 中使用 pcntl_fork() 期望完成并行操作不是一个合理的行为。</p>
<h1 id="TL-DR"><a href="#TL-DR" class="headerlink" title="TL;DR"></a>TL;DR</h1><p>在 PHP-FPM 中使用 pcntl 扩展在一定情况下是可行的,但是并不是合理的。</p>
使用Docker运行基于Nginx+PHP的应用程序
http://www.liaoaoyang.com/articles/2018/02/28/host-nginx-php-based-app-in-docker/
2018-02-28T03:46:19.000Z
2018-04-12T15:25:36.000Z
<h1 id="TL-DR"><a href="#TL-DR" class="headerlink" title="TL;DR"></a>TL;DR</h1><p>Docker作为目前主流的容器技术之一,可以更高效的利用系统资源,拥有更快的启动时间,提供一致的运行环境,能更轻松的维护和扩展。</p>
<p>作为开发者,在这项技术出现多年之后,是时候将他用于加速自己的App的开发、测试、部署阶段了。</p>
<p>本文是个人的学习笔记,目的在于描述如何让一个基于 Nginx + PHP 的应用程序在 Docker 中运行起来。原理性的内容会另起篇幅。</p>
<a id="more"></a>
<h1 id="运行环境"><a href="#运行环境" class="headerlink" title="运行环境"></a>运行环境</h1><h2 id="OS"><a href="#OS" class="headerlink" title="OS"></a>OS</h2><p>使用 <code>Ubuntu 16.04 LTS</code> 完成,同样也可以选用 <code>CentOS 7</code>,因为问题较多,放弃了在 <code>CentOS 6.5</code> 上的尝试(毕竟是很老的系统了)。</p>
<h2 id="Docker"><a href="#Docker" class="headerlink" title="Docker"></a>Docker</h2><p>1.13</p>
<h2 id="Nginx"><a href="#Nginx" class="headerlink" title="Nginx"></a>Nginx</h2><p>选用了比较喜欢的 <code>Openresty</code>,版本为 <code>1.13.6.1</code>,镜像为 <code>openresty/openresty:alpine</code>。</p>
<h2 id="PHP"><a href="#PHP" class="headerlink" title="PHP"></a>PHP</h2><p>本文为 <code>7.2.2</code>,镜像为 <code>php:7.2.2-fpm-alpine3.7</code>。</p>
<h1 id="部署"><a href="#部署" class="headerlink" title="部署"></a>部署</h1><p>如果要用 Docker 运行自己的应用程序,首先要对镜像有一定的了解。</p>
<h2 id="openresty-容器"><a href="#openresty-容器" class="headerlink" title="openresty 容器"></a>openresty 容器</h2><p>对于 OpenResty 来说,Dockerfile 需要了解的主要是 Nginx 的配置。</p>
<p>从 Dockerfile 中可以看到 nginx.conf 是在构建时拷贝到容器之中的。</p>
<figure class="highlight stata"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">COPY</span> nginx.<span class="keyword">conf</span> /usr/<span class="keyword">local</span>/openresty/nginx/<span class="keyword">conf</span>/nginx.<span class="keyword">conf</span></span><br></pre></td></tr></table></figure>
<p>在 GitHub 上可以看到 <a href="https://github.com/openresty/docker-openresty/blob/master/nginx.conf" target="_blank" rel="external">nginx.conf</a> 文件的内容:</p>
<figure class="highlight nginx"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#user nobody;</span></span><br><span class="line"><span class="title">worker_processes</span> <span class="number">1</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">#error_log logs/error.log;</span></span><br><span class="line"><span class="comment">#error_log logs/error.log notice;</span></span><br><span class="line"><span class="comment">#error_log logs/error.log info;</span></span><br><span class="line"></span><br><span class="line"><span class="comment">#pid logs/nginx.pid;</span></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="title">events</span> {</span><br><span class="line"> <span class="title">worker_connections</span> <span class="number">1024</span>;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="title">http</span> {</span><br><span class="line"> <span class="title">include</span> mime.types;</span><br><span class="line"> <span class="title">default_type</span> application/octet-stream;</span><br><span class="line"></span><br><span class="line"> <span class="comment">#log_format main '$remote_addr - $remote_user [$time_local] "$request" '</span></span><br><span class="line"> <span class="comment"># '$status $body_bytes_sent "$http_referer" '</span></span><br><span class="line"> <span class="comment"># '"$http_user_agent" "$http_x_forwarded_for"';</span></span><br><span class="line"></span><br><span class="line"> <span class="comment">#access_log logs/access.log main;</span></span><br><span class="line"></span><br><span class="line"> <span class="title">sendfile</span> <span class="built_in">on</span>;</span><br><span class="line"> <span class="comment">#tcp_nopush on;</span></span><br><span class="line"></span><br><span class="line"> <span class="comment">#keepalive_timeout 0;</span></span><br><span class="line"> <span class="title">keepalive_timeout</span> <span class="number">65</span>;</span><br><span class="line"></span><br><span class="line"> <span class="comment">#gzip on;</span></span><br><span class="line"></span><br><span class="line"> <span class="title">include</span> /etc/nginx/conf.d/<span class="regexp">*.conf</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<p>可以了解到如下信息:</p>
<ul>
<li>Nginx 配置文件为 /usr/local/openresty/nginx/conf/nginx.conf</li>
<li>vhosts配置文件目录为 /etc/nginx/conf.d/,这一个目录可以放置需要运行的项目的Nginx配置文件</li>
</ul>
<h2 id="php-fpm-容器"><a href="#php-fpm-容器" class="headerlink" title="php-fpm 容器"></a>php-fpm 容器</h2><p>对于 PHP 来说,Dockerfile 需要了解的至少有如下几个方面:</p>
<ul>
<li>php/php-fpm/php-config 等二进制文件的安装位置</li>
<li>php.ini 文件的位置以及内容</li>
<li>php-fpm.conf 文件的位置以及内容</li>
<li>php-fpm 运行用户与用户组</li>
</ul>
<p>Docker 在构建镜像时,通过 Dockerfile 对构建镜像的动作进行描述。可以通过分析一下这个文件了解一下选用镜像的特点。</p>
<h3 id="php-7-2-2-fpm-alpine3-7-Dockerfile"><a href="#php-7-2-2-fpm-alpine3-7-Dockerfile" class="headerlink" title="php:7.2.2-fpm-alpine3.7 Dockerfile"></a>php:7.2.2-fpm-alpine3.7 Dockerfile</h3><p><a href="https://github.com/docker-library/php/blob/b4e5b2b22ae2e8de03e882a2666d6e39a7c79f93/7.2/alpine3.7/fpm/Dockerfile" target="_blank" rel="external">源文件</a> 内容虽然多,但是需要关注的东西并不多,带着之前的问题去阅读这个文件。</p>
<p>这个文件事实上完成的主要是如下几个工作:</p>
<ul>
<li>声明依赖的基础镜像</li>
<li>设定环境变量</li>
<li>拷贝 PHP 环境所需的特定工具(如扩展安装工具)</li>
<li>构建以及配置 PHP 7.2.2</li>
<li>启动 php-fpm 以及暴露端口</li>
</ul>
<h3 id="php-php-fpm-php-config-等二进制文件的安装位置"><a href="#php-php-fpm-php-config-等二进制文件的安装位置" class="headerlink" title="php/php-fpm/php-config 等二进制文件的安装位置"></a>php/php-fpm/php-config 等二进制文件的安装位置</h3><p>在整个文件中没有看到诸如 <code>--prefix</code> 这样的参数,那么可以得知会把这些文件默认安装到 <code>/usr/local/bin/</code> 目录下。</p>
<p>启动一个容器进行验证:</p>
<figure class="highlight stata"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"> ➜ docker <span class="keyword">run</span> -it --<span class="keyword">rm</span> php:7.2.2-fpm-alpine3.7 /bin/<span class="keyword">sh</span></span><br><span class="line">/<span class="keyword">var</span>/www/html # <span class="keyword">ls</span> /usr/<span class="keyword">local</span>/bin/</span><br><span class="line">docker-php-entrypoint docker-php-ext-install peardev phar.phar phpdbg</span><br><span class="line">docker-php-ext-configure docker-php-source pecl php phpize</span><br><span class="line">docker-php-ext-enable pear phar php-config</span><br></pre></td></tr></table></figure>
<h3 id="php-ini-文件的位置以及内容"><a href="#php-ini-文件的位置以及内容" class="headerlink" title="php.ini 文件的位置以及内容"></a>php.ini 文件的位置以及内容</h3><p>文件37行:</p>
<figure class="highlight gradle"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">ENV PHP_INI_DIR <span class="regexp">/usr/</span>local<span class="regexp">/etc/</span>php</span><br></pre></td></tr></table></figure>
<p>在后续构建 configure 阶段(92行开始):</p>
<figure class="highlight livescript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br></pre></td><td class="code"><pre><span class="line">RUN set -xe <span class="string">\</span></span><br><span class="line"> && apk add --<span class="literal">no</span>-cache --virtual .build-deps <span class="string">\</span></span><br><span class="line"> $PHPIZE_DEPS <span class="string">\</span></span><br><span class="line"> coreutils <span class="string">\</span></span><br><span class="line"> curl-dev <span class="string">\</span></span><br><span class="line"> libedit-dev <span class="string">\</span></span><br><span class="line"> libressl-dev <span class="string">\</span></span><br><span class="line"> libxml2-dev <span class="string">\</span></span><br><span class="line"> sqlite-dev <span class="string">\</span></span><br><span class="line"> <span class="string">\</span></span><br><span class="line"> && <span class="keyword">export</span> CFLAGS=<span class="string">"$PHP_CFLAGS"</span> <span class="string">\</span></span><br><span class="line"> CPPFLAGS=<span class="string">"$PHP_CPPFLAGS"</span> <span class="string">\</span></span><br><span class="line"> LDFLAGS=<span class="string">"$PHP_LDFLAGS"</span> <span class="string">\</span></span><br><span class="line"> && docker-php-source extract <span class="string">\</span></span><br><span class="line"> && cd /usr/src/php <span class="string">\</span></span><br><span class="line"> && gnuArch=<span class="string">"$(dpkg-architecture --query DEB_BUILD_GNU_TYPE)"</span> <span class="string">\</span></span><br><span class="line"> && ./configure <span class="string">\</span></span><br><span class="line"> --build=<span class="string">"$gnuArch"</span> <span class="string">\</span></span><br><span class="line"> --<span class="keyword">with</span>-config-file-path=<span class="string">"$PHP_INI_DIR"</span> <span class="string">\</span></span><br><span class="line"> --<span class="keyword">with</span>-config-file-scan-dir=<span class="string">"$PHP_INI_DIR/conf.d"</span> <span class="string">\</span></span><br></pre></td></tr></table></figure>
<p>可以看到也定义了这一个目录。</p>
<p>然而在整个文件中也没有看到拷贝 ini 文件到对应目录的操作,所以这个容器是没有使用 php.ini 文件进行配置的。</p>
<p>如果想要知道当前默认的配置项,不妨进入容器,通过 <code>/usr/local/bin/php -i</code> 进行查看。</p>
<h3 id="php-fpm-conf-文件的位置以及内容"><a href="#php-fpm-conf-文件的位置以及内容" class="headerlink" title="php-fpm.conf 文件的位置以及内容"></a>php-fpm.conf 文件的位置以及内容</h3><p>类似 php.ini 文件的情况。</p>
<h3 id="php-fpm-运行用户与用户组"><a href="#php-fpm-运行用户与用户组" class="headerlink" title="php-fpm 运行用户与用户组"></a>php-fpm 运行用户与用户组</h3><p>这里涉及到一个权限的问题,Docker 可以把应用代码拷贝到容器中,但是也可以通过挂载目录的方式完成,将目录的 owner 设定为当前容器运行的 uid 可以避免权限带来问题。</p>
<p>文件29行可以看到新增了 uid 为 82 的 <code>www-data</code> 用户。</p>
<figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">RUN <span class="operator"><span class="keyword">set</span> -x \</span><br><span class="line"> && addgroup -<span class="keyword">g</span> <span class="number">82</span> -S www-<span class="keyword">data</span> \</span><br><span class="line"> && adduser -u <span class="number">82</span> -<span class="keyword">D</span> -S -<span class="keyword">G</span> www-<span class="keyword">data</span> www-<span class="keyword">data</span></span></span><br></pre></td></tr></table></figure>
<p>在构建阶段,也声明了 fpm 运行者为 <code>www-data</code>:</p>
<figure class="highlight brainfuck"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">ENV</span> <span class="comment">PHP_EXTRA_CONFIGURE_ARGS</span> <span class="literal">-</span><span class="literal">-</span><span class="comment">enable</span><span class="literal">-</span><span class="comment">fpm</span> <span class="literal">-</span><span class="literal">-</span><span class="comment">with</span><span class="literal">-</span><span class="comment">fpm</span><span class="literal">-</span><span class="comment">user=www</span><span class="literal">-</span><span class="comment">data</span> <span class="literal">-</span><span class="literal">-</span><span class="comment">with</span><span class="literal">-</span><span class="comment">fpm</span><span class="literal">-</span><span class="comment">group=www</span><span class="literal">-</span><span class="comment">data</span></span><br></pre></td></tr></table></figure>
<h2 id="启动容器"><a href="#启动容器" class="headerlink" title="启动容器"></a>启动容器</h2><h3 id="通过-link-关联-openresty-与-php-fpm"><a href="#通过-link-关联-openresty-与-php-fpm" class="headerlink" title="通过 link 关联 openresty 与 php-fpm"></a>通过 link 关联 openresty 与 php-fpm</h3><p>Docker 容器之间不能直接访问,可以用 link 的方式,也可以直接把端口映射到主机端口上,通过主机的网络进行通信。</p>
<p>需要先行启动 php-fpm 容器。</p>
<h4 id="启动-php-fpm-容器"><a href="#启动-php-fpm-容器" class="headerlink" title="启动 php-fpm 容器"></a>启动 php-fpm 容器</h4><p>因为需要配置诸如时区等配置,然而当前容器又没有默认的 php.ini 文件,可以挂载一个自行配置 php.ini 文件。</p>
<p><code>php.ini</code> 配置文件中将 PHP 错误日志目录定义为 <code>/var/www/logs/php_error.log</code>,所以挂载可以写入日志的目录到容器中的 <code>/var/www/logs</code> 目录下。</p>
<figure class="highlight groovy"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">docker run --name=test_phpfpm \</span><br><span class="line">-v <span class="regexp">/data1/</span>www<span class="regexp">/etc/</span>test<span class="regexp">/conf/</span>php<span class="regexp">/php.ini:/</span>usr<span class="regexp">/local/</span>etc<span class="regexp">/php/</span>php.<span class="string">ini:</span>ro \</span><br><span class="line">-v <span class="regexp">/data1/</span>www<span class="regexp">/etc/</span>test<span class="regexp">/conf/</span><span class="string">php:</span><span class="regexp">/usr/</span>local<span class="regexp">/etc/</span>php<span class="regexp">/conf.d/</span>:ro \</span><br><span class="line">-v <span class="regexp">/data1/</span>www<span class="regexp">/htdocs/</span><span class="string">test:</span><span class="regexp">/var/</span>www/html \</span><br><span class="line">-v <span class="regexp">/data1/</span>www<span class="regexp">/logs:/</span>var<span class="regexp">/www/</span>logs \</span><br><span class="line">-d <span class="string">php:</span><span class="number">7.2</span><span class="number">.2</span>-fpm-alpine3<span class="number">.7</span></span><br></pre></td></tr></table></figure>
<p>php-fpm 的容器名为 <code>test_phpfpm</code> ,与之连接的其他容器可以通过这个名字直接访问容器。</p>
<h4 id="启动-openresty-容器"><a href="#启动-openresty-容器" class="headerlink" title="启动 openresty 容器"></a>启动 openresty 容器</h4><figure class="highlight groovy"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">docker run --link=test_phpfpm --name=test_openresty -p <span class="number">8808</span>:<span class="number">80</span> \</span><br><span class="line">-v <span class="regexp">/data1/</span>www<span class="regexp">/etc/</span>test<span class="regexp">/conf/</span>nginx<span class="regexp">/nginx.test.config:/</span>usr<span class="regexp">/local/</span>openresty<span class="regexp">/nginx/</span>conf/nginx.<span class="string">conf:</span>ro \</span><br><span class="line">-v <span class="regexp">/data1/</span>www<span class="regexp">/etc/</span>test<span class="regexp">/conf/</span><span class="string">nginx:</span><span class="regexp">/etc/</span>nginx<span class="regexp">/conf.d/</span>:ro \</span><br><span class="line">-v <span class="regexp">/data1/</span>www<span class="regexp">/logs:/</span>usr<span class="regexp">/local/</span>openresty<span class="regexp">/nginx/</span>logs \</span><br><span class="line">-v <span class="regexp">/data1/</span>www<span class="regexp">/htdocs/</span><span class="string">test:</span><span class="regexp">/var/</span>www/html \</span><br><span class="line">-d openresty/<span class="string">openresty:</span>alpine</span><br></pre></td></tr></table></figure>
<p>对应的 Nginx 配置文件为:</p>
<figure class="highlight nginx"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br></pre></td><td class="code"><pre><span class="line"><span class="title">worker_processes</span> <span class="number">1</span>;</span><br><span class="line"></span><br><span class="line"><span class="title">events</span> {</span><br><span class="line"> <span class="title">worker_connections</span> <span class="number">65535</span>;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="title">http</span> {</span><br><span class="line"> <span class="title">include</span> mime.types;</span><br><span class="line"> <span class="comment">#default_type application/octet-stream;</span></span><br><span class="line"> <span class="title">default_type</span> text/html;</span><br><span class="line"></span><br><span class="line"> <span class="title">sendfile</span> <span class="built_in">on</span>;</span><br><span class="line"></span><br><span class="line"> <span class="title">keepalive_timeout</span> <span class="number">65</span>;</span><br><span class="line"></span><br><span class="line"> <span class="title">server</span> {</span><br><span class="line"> <span class="title">listen</span> <span class="number">80</span> default_server;</span><br><span class="line"> <span class="title">root</span> /var/www/html;</span><br><span class="line"> <span class="title">index</span> index.php index.html index.htm;</span><br><span class="line"></span><br><span class="line"> <span class="title">location</span> / {</span><br><span class="line"> <span class="title">try_files</span> <span class="variable">$uri</span> <span class="variable">$uri</span>/ /index.php?<span class="variable">$query_string</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="title">location</span> <span class="regexp">~ \.php($|/)</span> {</span><br><span class="line"> <span class="title">fastcgi_pass</span> test_phpfpm:<span class="number">9000</span>;</span><br><span class="line"> <span class="title">fastcgi_index</span> index.php;</span><br><span class="line"> <span class="title">include</span> fastcgi_params;</span><br><span class="line"> <span class="title">set</span> <span class="variable">$path_info</span> <span class="string">""</span>;</span><br><span class="line"> <span class="title">set</span> <span class="variable">$real_script_name</span> <span class="variable">$fastcgi_script_name</span>;</span><br><span class="line"></span><br><span class="line"> <span class="title">if</span> (<span class="variable">$fastcgi_script_name</span> <span class="regexp">~ "^(.+?\.php)(/.+)$")</span> {</span><br><span class="line"> <span class="title">set</span> <span class="variable">$real_script_name</span> <span class="variable">$1</span>;</span><br><span class="line"> <span class="title">set</span> <span class="variable">$path_info</span> <span class="variable">$2</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="title">fastcgi_param</span> SCRIPT_FILENAME <span class="variable">$document_root</span><span class="variable">$real_script_name</span>;</span><br><span class="line"> <span class="title">fastcgi_param</span> SCRIPT_NAME <span class="variable">$real_script_name</span>;</span><br><span class="line"> <span class="title">fastcgi_param</span> PATH_INFO <span class="variable">$path_info</span>;</span><br><span class="line"> <span class="title">fastcgi_param</span> PHP_VALUE open_basedir=<span class="variable">$document_root</span>:/tmp/:/proc/:/dev/urandom;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="title">include</span> /etc/nginx/conf.d/<span class="regexp">*.conf</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<p>至此,在宿主机 <code>/data1/www/htdocs/test</code> 目录下的 PHP 项目可以对外提供服务了。</p>
<h1 id="TL-DR"><a href="#TL-DR" class="headerlink" title="TL;DR"></a>TL;DR</h1><p>Docker作为目前主流的容器技术之一,可以更高效的利用系统资源,拥有更快的启动时间,提供一致的运行环境,能更轻松的维护和扩展。</p>
<p>作为开发者,在这项技术出现多年之后,是时候将他用于加速自己的App的开发、测试、部署阶段了。</p>
<p>本文是个人的学习笔记,目的在于描述如何让一个基于 Nginx + PHP 的应用程序在 Docker 中运行起来。原理性的内容会另起篇幅。</p>
《垃圾回收的算法与实现》
http://www.liaoaoyang.com/articles/2018/01/31/note-of-garbage-collection-algorithms-and-implementations/
2018-01-31T13:00:00.000Z
2018-01-31T13:10:35.000Z
<h1 id="TL-DR"><a href="#TL-DR" class="headerlink" title="TL;DR"></a>TL;DR</h1><p><a href="https://book.douban.com/subject/26821357/" target="_blank" rel="external">《垃圾回收的算法与实现》</a>一书的读书笔记。</p>
<a id="more"></a>
<h1 id="笔记"><a href="#笔记" class="headerlink" title="笔记"></a>笔记</h1><h2 id="第1章-学习GC之前"><a href="#第1章-学习GC之前" class="headerlink" title="第1章 学习GC之前"></a>第1章 学习GC之前</h2><h3 id="GC-是什么"><a href="#GC-是什么" class="headerlink" title="GC 是什么"></a>GC 是什么</h3><p>GC实际上可以看做管理堆上内存中对象的一个应用程序。</p>
<p>GC主要做清理工作,然而GC的实现会影响新对象的分配。</p>
<h3 id="对象的头部以及域"><a href="#对象的头部以及域" class="headerlink" title="对象的头部以及域"></a>对象的头部以及域</h3><p>对象分为<code>头部</code>和<code>域</code>两个部分,<code>头部</code>包含对象的基础信息以及GC相关的信息,<code>域</code>则是对象使用者可以直接操作的部分。如果对应到 PHP 上(参见<code>zend_gc.h</code>),在生成 ZVAL 时,实际上会申请一个 <code>zval_gc_info</code> 大小的空间:</p>
<figure class="highlight crystal"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line">typedef <span class="class"><span class="keyword">struct</span> <span class="title">_zval_gc_info</span> {</span></span><br><span class="line"> zval z;</span><br><span class="line"> <span class="class"><span class="keyword">union</span> {</span></span><br><span class="line"> gc_root_buffer *buffered;</span><br><span class="line"> <span class="class"><span class="keyword">struct</span> <span class="title">_zval_gc_info</span> *<span class="title">next</span>;</span></span><br><span class="line"> } u;</span><br><span class="line">} zval_gc_info;</span><br><span class="line"></span><br><span class="line"><span class="regexp">//</span> ...</span><br><span class="line"></span><br><span class="line">/* <span class="constant">The</span> following macros override macros from zend_alloc.h *<span class="regexp">/</span><br><span class="line">#undef ALLOC_ZVAL</span><br><span class="line">#define ALLOC_ZVAL(z) \</span><br><span class="line"> do { \</span><br><span class="line"> (z) = (zval*)emalloc(sizeof(zval_gc_info)); \</span><br><span class="line"> GC_ZVAL_INIT(z); \</span><br><span class="line"> } while (0)</span></span><br></pre></td></tr></table></figure>
<p>那么对于上述结构来说,头部应该是名为<code>u</code>的联合体以及<code>zval</code>中关于引用计数的部分,<code>zval z</code>则是对象的域。</p>
<h3 id="GC算法的评价标准"><a href="#GC算法的评价标准" class="headerlink" title="GC算法的评价标准"></a>GC算法的评价标准</h3><h4 id="常用的四大标准"><a href="#常用的四大标准" class="headerlink" title="常用的四大标准"></a>常用的四大标准</h4><ul>
<li>吞吐量,即单位时间内GC处理的内存大小</li>
<li>最大暂停时间,即GC会中断程序正常执行的时间</li>
<li>堆的使用效率,即内存的使用方式以及头信息的占用比例</li>
<li>访问的局部性,即是否能更好的利用高速寄存器</li>
</ul>
<h2 id="第2章-GC标记-清除算法"><a href="#第2章-GC标记-清除算法" class="headerlink" title="第2章 GC标记-清除算法"></a>第2章 GC标记-清除算法</h2><p><code>标记-清除</code>算法的核心是从根出发,通过 DFS 逐个标记活动的对象,标记完成之后遍历堆,回收不活动的对象。</p>
<p>这一算法可能会有如下的一些问题:</p>
<ul>
<li>碎片化引起分配操作时延增大</li>
<li>不兼容写时拷贝(标记内容位于堆中,执行标记动作时触发不必要的复制操作)</li>
</ul>
<p>碎片化带来的问题是找到合适内存空间可能需要较长的线性查找操作时间。</p>
<p>为了能减低清除操作带来的时间花费,会把清理工作推迟到需要分配新空间时进行,但是可能会影响分配的速度。</p>
<h3 id="碎片化问题的权衡"><a href="#碎片化问题的权衡" class="headerlink" title="碎片化问题的权衡"></a>碎片化问题的权衡</h3><p>碎片化问题,<code>标记-清除</code>算法的解决思路是通过对堆中内存分类进行处理,可以是通过划分多个大小的堆的方式,只允许在特定堆上申请空间,或者是 BiBOP(Big Bag Of Pages) 方法,将堆划分为多个块,每个块只能申请特定大小的空间。</p>
<h3 id="写时拷贝问题"><a href="#写时拷贝问题" class="headerlink" title="写时拷贝问题"></a>写时拷贝问题</h3><p>为了兼容这一个问题,通过独立设定标记位的形式,管理堆内存,只处理不在堆上的标记位。</p>
<h2 id="第3章-引用计数法"><a href="#第3章-引用计数法" class="headerlink" title="第3章 引用计数法"></a>第3章 引用计数法</h2><p>引用计数法的特点在于内存空间的管理和对象的管理同时进行。</p>
<p>引用计数有:</p>
<ul>
<li>即刻回收垃圾</li>
<li>缩短了最大暂停时间</li>
<li>不需要大规模的指针操作</li>
</ul>
<p>这些优点。</p>
<p>但是也有:</p>
<ul>
<li>计数器操作繁重</li>
<li>降低内存利用率</li>
<li>实现繁琐</li>
<li>无法解决循环引用问题</li>
</ul>
<p>等缺点,不过上述部分问题都有成熟的应对方案。</p>
<h3 id="计数器操作繁重"><a href="#计数器操作繁重" class="headerlink" title="计数器操作繁重"></a>计数器操作繁重</h3><p>通过延迟引用计数法解决根引用计数操作频繁的问题。</p>
<p>核心是预留出ZCT(Zero Count Table),当一个对象引用计数到达0时加入ZCT,之后在ZCT满之后对根可达的对象引用计数增加,之后在清理引用计数为0的对象。</p>
<p>但是这个会带来的问题是ZCT的大小阈值设定问题,过小频繁触发ZCT扫描,过大导致扫描时长过长。</p>
<h3 id="降低内存使用率"><a href="#降低内存使用率" class="headerlink" title="降低内存使用率"></a>降低内存使用率</h3><h4 id="Sticky引用计数法"><a href="#Sticky引用计数法" class="headerlink" title="Sticky引用计数法"></a>Sticky引用计数法</h4><p>可以通过Sticky引用计数法,减少用于计数的数据存储空间(研究证明很少有对象有超过32次引用,即在5个bit的存储空间的情况下),之后通过改进后的<code>标记-清除</code>算法进行GC操作,无论是否因为计数器溢出,总能进行GC操作。</p>
<p>改进后的<code>标记-清除</code>算法主要的区别在于:</p>
<ul>
<li>先将所有对象引用计数设置为0</li>
<li>从根引用的对象出发,使用堆栈记录子对象,并且每个对象保证只进入堆栈一次的前提下,逐个增加引用计数</li>
<li>清理上述递归过程后引用计数仍然为0对象</li>
</ul>
<p>上述操作完成之后可以处理循环引用问题,简单思考一下:如果A对象引用B对象,B对象引用A对象,根据引用计数原理,二者虽然没有其他对象引用,但是因为计数都为1,所以无法被回收。通过上述方法首先引用计数置0之后,由于从根开始,发现没有任何可达的路径,那么这两个对象最后的引用计数为0,可以被GC。</p>
<p>溢出导致引用计数变为0不会影响GC的原因在于从根出发,如果一个对象是垃圾,那么必定没有可以到达的路径,也就是无法将引用计数增加,即最终引用计数为0。但是如果只是因为溢出,对象总有路径可达,所以不会引起无法GC的问题。</p>
<h3 id="循环引用问题"><a href="#循环引用问题" class="headerlink" title="循环引用问题"></a>循环引用问题</h3><p>在<code>Sticky引用计数法</code>中提到可以通过<code>标记-清除</code>算法可以解决循环引用问题,因为遍历整个堆的过程,所以效率会比较底下,所以充满智慧的前人Rafael D.Lins研究出了<code>部分标记-清除</code>算法解决了这个问题。</p>
<p>和常规的<code>标记-清除</code>算法有所不同的是,<code>部分标记-清除</code>算法关注非活动对象(<code>标记-清除</code>算法会找出所有的活动或者说可达的对象)。</p>
<p><code>部分标记-清除</code>算法的核心在<em>关注非活动对象</em>,即只是进行限定范围内的搜索。对于引用计数直接置为0的对象,是引用计数法的经典情况,当然可以直接进行GC操作,进行回收。但是在引用计数-1之后,引用计数仍然大于0的,会认为疑似是循环引用。最常见的情况,如A引用B,B引用A,C引用B:</p>
<figure class="highlight nimrod"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">+----------+ +----------+ +----------+</span><br><span class="line">| | --> | | --> | |</span><br><span class="line">| <span class="type">Object</span> C | | <span class="type">Object</span> B | | <span class="type">Object</span> A |</span><br><span class="line">| | | <span class="keyword">ref</span>:<span class="number">2</span> | <-- | <span class="keyword">ref</span>:<span class="number">1</span> |</span><br><span class="line">+----------+ +----------+ +----------+</span><br></pre></td></tr></table></figure>
<p>那么A引用计数为1,B引用计数为2,在回收C之后,发现B的引用计数仍然为1,可以怀疑B存在循环引用情况:</p>
<figure class="highlight nimrod"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">+----------+ +----------+ +----------+</span><br><span class="line">| | -X-> | | --> | |</span><br><span class="line">| <span class="type">Object</span> C | | <span class="type">Object</span> B | | <span class="type">Object</span> A |</span><br><span class="line">| <span class="type">GC</span> ed | | <span class="keyword">ref</span>:<span class="number">1</span> | <-- | <span class="keyword">ref</span>:<span class="number">1</span> |</span><br><span class="line">+----------+ +----------+ +----------+</span><br></pre></td></tr></table></figure>
<p>首先要明确几个颜色的概念(即对象的状态):</p>
<ol>
<li>黑(BLACK):绝对不是垃圾的对象(对象产生时的初始颜色) </li>
<li>白(WHITE):绝对是垃圾的对象</li>
<li>灰(GRAY):搜索完毕的对象</li>
<li>阴影(HATCH):可能是循环垃圾的对象</li>
</ol>
<p>对于上述问题,<code>部分标记-清除</code>算法采用的是在引用计数减少时,将疑似垃圾对象入队列,并记录对象颜色为阴影状态<code>HATCH</code>,如果已经是<code>HATCH</code>了,说明对象已经是在队列之中,不用重复入队列。</p>
<p>在需要进行GC操作时,从队列中扫描所有阴影(HATCH)或者黑色(BLACK)的对象,将这些对象的<strong>子对象</strong>引用计数-1,并且将<strong>当前对象</strong>颜色标记为灰色(GRAY),即已经访问过的对象,防止重复操作,对于对象的子对象,做同样的处理,因为子对象是从队列触达的,有可能并不在队列之中,也就是没有标记成HATCH,而是BLACK这样的正常状态,因为是一个递归调用,所以在最开始的描述中,会告知可以处理黑色的对象。</p>
<p>队列在此处的作用相当重要:<strong>队列保留了对象一旦从根切断访问路径之后仍能被GC程序访问到的唯一路径</strong>。</p>
<p>之后对队列中所有的灰色(GRAY)对象,对所有引用计数仍然大于0的对象,认定不存在循环引用问题,尝试涂为黑色(BLACK)并+1引用计数,对于黑色对象的不为黑色的子对象也做同样的操作;否则将对象标记为白色(WHITE),同样的,对于白色对象的子对象也做一样的操作。</p>
<p>最终,回收被标记为白色(WHITE)的所有对象。</p>
<p>看到这里,PHPer会不会突然想起,这不很类似PHP5.3之后的GC实现方式吗?从<a href="http://www.php-internals.com/book/?p=chapt06/06-04-01-new-garbage-collection" target="_blank" rel="external">TIPI</a>中可以读到,只不过是把HATCH用PURPLE这一颜色替代了。</p>
<p>来看如下例子:</p>
<figure class="highlight nimrod"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">+----------+ +----------+ +----------+</span><br><span class="line">| | ---> | <span class="type">BLACK</span> | --> | <span class="type">BLACK</span> |</span><br><span class="line">| <span class="type">ROOT</span> | | <span class="type">Object</span> B | | <span class="type">Object</span> A |</span><br><span class="line">| | | <span class="keyword">ref</span>:<span class="number">2</span> | <-- | <span class="keyword">ref</span>:<span class="number">1</span> |</span><br><span class="line">+----------+ +----------+ +----------+</span><br></pre></td></tr></table></figure>
<p>对于对象B来说,如果执行 <code>unset($b)</code> 操作,此时引用计数会-1,但是对象B的引用计数目前仍然为1,符合我们对疑似循环引用的推断:</p>
<figure class="highlight nimrod"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">+----------+ +----------+ +----------+</span><br><span class="line">| | -X-> | <span class="type">HATCH</span> | --> | <span class="type">BLACK</span> |</span><br><span class="line">| <span class="type">ROOT</span> | | <span class="type">Object</span> B | | <span class="type">Object</span> A |</span><br><span class="line">| | | <span class="keyword">ref</span>:<span class="number">1</span> | <-- | <span class="keyword">ref</span>:<span class="number">1</span> |</span><br><span class="line">+----------+ +----------+ +----------+</span><br></pre></td></tr></table></figure>
<p>如果按照常规的引用计数的处理,目前对象B引用计数仍然大于0,是不能被回收的。</p>
<p>对象B入HATCH队列,假设开始GC操作,首先执行灰色标记操作:</p>
<figure class="highlight nimrod"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">+----------+ +----------+ +----------+</span><br><span class="line">| | -X-> | <span class="type">GRAY</span> | --> | <span class="type">BLACK</span> |</span><br><span class="line">| <span class="type">ROOT</span> | | <span class="type">Object</span> B | | <span class="type">Object</span> A |</span><br><span class="line">| | | <span class="keyword">ref</span>:<span class="number">1</span> | <-- | <span class="keyword">ref</span>:<span class="number">0</span> |</span><br><span class="line">+----------+ +----------+ +----------+</span><br></pre></td></tr></table></figure>
<p>根据算法,对于对象B的子对象对象A也需要进行标记操作:</p>
<figure class="highlight nimrod"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">+----------+ +----------+ +----------+</span><br><span class="line">| | -X-> | <span class="type">GRAY</span> | --> | <span class="type">GRAY</span> |</span><br><span class="line">| <span class="type">ROOT</span> | | <span class="type">Object</span> B | | <span class="type">Object</span> A |</span><br><span class="line">| | | <span class="keyword">ref</span>:<span class="number">1</span> | <-- | <span class="keyword">ref</span>:<span class="number">0</span> |</span><br><span class="line">+----------+ +----------+ +----------+</span><br></pre></td></tr></table></figure>
<p>根据算法,对于对象A的子对象对象B也需要进行标记操作,但是因为对象B已经标记为灰色,所以无需进行标记操作,但是仍然需要进行子对象引用计数-1操作,下一步执行扫描灰色对象操作:</p>
<figure class="highlight nimrod"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">+----------+ +----------+ +----------+</span><br><span class="line">| | -X-> | <span class="type">WHITE</span> | --> | <span class="type">WHITE</span> |</span><br><span class="line">| <span class="type">ROOT</span> | | <span class="type">Object</span> B | | <span class="type">Object</span> A |</span><br><span class="line">| | | <span class="keyword">ref</span>:<span class="number">0</span> | <-- | <span class="keyword">ref</span>:<span class="number">0</span> |</span><br><span class="line">+----------+ +----------+ +----------+</span><br></pre></td></tr></table></figure>
<p>不幸的是,对象B作为一个灰色对象并且引用计数已经为0,需要标记成白色,确认是一个垃圾对象。根据算法也要对子对象进行同样的操作,所以对象A也会标记成白色,因为当前没有存在黑色的对象,所以标记黑色操作不进行。</p>
<p>最后回收所有白色对象。</p>
<p>对于需要对<strong>子对象</strong>引用计数先-1,而不是对象本身-1,如果首先对对象本身-1,那么假设对象A当前引用计数为2,存在子对象B,子对象B没有引用A。现在从根对象中解除引用,A的引用计数变为1,在标记操作过程中会使得A对象引用计数边为0,而B对象引用计数也会变为0,但是事实上,B并没有引用A,不存在循环引用情况,但是这样的操作步骤却使得A误认为是垃圾,被清理。</p>
<p>来看一下PHP中的标记阶段 <code>zobj_mark_grey()</code> 函数:</p>
<figure class="highlight zephir"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">static</span> void zobj_mark_grey(struct _store_object *obj, zval *pz TSRMLS_DC)</span><br><span class="line">{</span><br><span class="line"> Bucket *p;</span><br><span class="line"> zend_object_get_gc_t get_gc;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (GC_GET_COLOR(obj->buffered) != GC_GREY) {</span><br><span class="line"> GC_BENCH_INC(zobj_marked_grey);</span><br><span class="line"> GC_SET_COLOR(obj->buffered, GC_GREY);</span><br><span class="line"> <span class="keyword">if</span> (EXPECTED(EG(objects_store).object_buckets[Z_OBJ_HANDLE_P(pz)].valid &&</span><br><span class="line"> (get_gc = Z_OBJ_HANDLER_P(pz, get_gc)) != <span class="keyword">NULL</span>)) {</span><br><span class="line"> <span class="keyword">int</span> i, n;</span><br><span class="line"> zval **table;</span><br><span class="line"> HashTable *props = get_gc(pz, &table, &n TSRMLS_CC);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> (i = <span class="number">0</span>; i < n; i++) {</span><br><span class="line"> <span class="keyword">if</span> (table[i]) {</span><br><span class="line"> pz = table[i];</span><br><span class="line"> <span class="keyword">if</span> (Z_TYPE_P(pz) != IS_ARRAY || Z_ARRVAL_P(pz) != &EG(symbol_table)) {</span><br><span class="line"> pz->refcount__gc--;</span><br><span class="line"> }</span><br><span class="line"> zval_mark_grey(pz TSRMLS_CC);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (!props) {</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"> p = props->pListHead;</span><br><span class="line"> <span class="keyword">while</span> (p != <span class="keyword">NULL</span>) {</span><br><span class="line"> pz = *(zval**)p->pData;</span><br><span class="line"> <span class="keyword">if</span> (Z_TYPE_P(pz) != IS_ARRAY || Z_ARRVAL_P(pz) != &EG(symbol_table)) {</span><br><span class="line"> pz->refcount__gc--;</span><br><span class="line"> }</span><br><span class="line"> zval_mark_grey(pz TSRMLS_CC);</span><br><span class="line"> p = p->pListNext;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<p>这里确实是对子对象先行操作引用计数的。</p>
<p>未完待续。</p>
<h1 id="TL-DR"><a href="#TL-DR" class="headerlink" title="TL;DR"></a>TL;DR</h1><p><a href="https://book.douban.com/subject/26821357/">《垃圾回收的算法与实现》</a>一书的读书笔记。</p>