如何屏蔽蜘蛛抓取

伟巴斯特 · 发表于 2013-2-25 02:56:36

如何屏蔽蜘蛛抓取“forum.php?”有这个动态链接地址呢？
Baidu 123.125.71.48 2013-5-24 10:38 http://www.bawu.cc/forum.php?mod=viewthread&tid=76&page=1&extra=
Baidu 123.125.71.77 2013-5-24 10:38 http://www.bawu.cc/forum.php?mod=post&action=edit&fid=37&tid=76&pid=174&page=1
Baidu 123.125.71.39 2013-5-24 10:36 http://www.bawu.cc/forum.php?mod=viewthread&tid=76&extra=page%3D1%26filter%3Dtypeid%26typeid%3D7%26t
Baidu 123.125.71.88 2013-5-24 10:31 http://www.bawu.cc/dianfei/t76/1
Baidu 123.125.71.35 2013-5-24 10:30 http://www.bawu.cc/jinhuaqingnian/t58/
Baidu 123.125.71.92 2013-5-24 10:25 http://www.bawu.cc/forum.php?mod=forumdisplay&fid=37&filter=typeid&typeid=8
Baidu 123.125.71.102 2013-5-24 10:24 http://www.bawu.cc/forum.php?mod=viewthread&tid=75&pid=173&page=1&extra=
Baidu 123.125.71.96 2013-5-24 10:24 http://www.bawu.cc/forum.php?mod=viewthread&tid=75&pid=172&page=1&extra=
Baidu 123.125.71.91 2013-5-24 10:23 http://www.bawu.cc/forum.php?mod=viewthread&tid=75&pid=171&page=1&extra=
Baidu 123.125.71.32 2013-5-24 10:23 http://www.bawu.cc/forum.php?mod=viewthread&tid=75&pid=170&page=1&extra=
Baidu 123.125.71.97 2013-5-24 10:23 http://www.bawu.cc/forum.php?mod=viewthread&tid=75&pid=169&page=1&extra=
Baidu 123.125.71.117 2013-5-24 10:22 http://www.bawu.cc/forum.php?mod=post&action=reply&fid=37&tid=75
Baidu 123.125.71.90 2013-5-24 10:21 http://www.bawu.cc/dianfei/t75/1
Baidu 123.125.71.33 2013-5-24 10:14 http://www.bawu.cc/forum.php?mod=viewthread&tid=74&page=1&extra=
Baidu 123.125.71.77 2013-5-24 10:14 http://www.bawu.cc/forum.php?mod=post&action=edit&fid=37&tid=74&pid=162&page=1