View Full Version : Fetch & Retrieve Adapter: vBulletin Forums
jstanden
11-08-2007, 02:10 AM
No idea on the IPB RSS, but I'll take a look. Edit: Their documentation indicates that they only have RSS export on forum posts. It might be possible to write a search rss feed for it, though.
Yeah, vBulletin is the same way. I'll have to write a 'Search Results as RSS' mod for it like I did for Mediawiki.
That's easy enough though, I just haven't had time to knock it off my list yet.
Looks like a lot of people are requesting the same thing for other uses:
http://www.vbulletin.org/forum/showthread.php?t=73170
With Mediawiki I actually used their API instead of just reading the database directly. I can likely do the same with vB, but their API has always been a bit crazy.
jstanden
11-11-2007, 01:44 AM
Here's a working prototype for searching a vBulletin forum by RSS. This plugs right into Fetch & Retrieve.
Just drop this file into your vBulletin root directory. Currently you must be using the FULLTEXT search method. This was written against VB 3.6.8, so if you're running a vastly different version you may need to tweak the query slightly.
Your Fetch & Retrieve URL would look like:
http://localhost/forums/searchrss.php?q=#find#
To restrict access to specific IPs (such as only your webserver for F&R requests), simply change:
if(1) {
To something like:
if($_SERVER['REMOTE_ADDR'] == '1.2.3.4') {
searchrss.php:
<?php
// RSS Search Results by WebGroup Media LLC (http://www.webgroupmedia.com/)
// MIT License
define("WGMRSS_TITLE", "Cerberus Helpdesk 4.0 Forums");
define("WGMRSS_BASEURL","http://www.cerb4.com/forums/");
error_reporting(E_ALL & ~E_NOTICE);
require_once('./global.php');
require_once(DIR . '/includes/functions_search.php');
require_once(DIR . '/includes/functions_forumlist.php');
require_once(DIR . '/includes/functions_misc.php');
if (1) {
$res = $db->query_read(sprintf("select p.postid,t.title,p.dateline,p.username,pp.pagetext _html,match(p.title,pagetext) against ('%s') as score from post p inner join thread t ON (t.threadid=p.threadid) inner join postparsed pp ON (pp.postid=p.postid) where match(p.title,pagetext) against ('%s') group by p.threadid order by score desc limit 0,100",
$db->escape_string($_REQUEST['q']),
$db->escape_string($_REQUEST['q'])
));
header('Content-type: text/xml');
?>
<rss xmlns:content="http://purl.org/rss/1.0/modules/content/" version="2.0">
<channel>
<title><![CDATA[<?php echo WGMRSS_TITLE; ?>]]></title>
<link><?php echo WGMRSS_BASEURL; ?></link>
<description></description>
<pubDate>Sun, 11 Nov 2007 00:56:50 +0000</pubDate>
<generator>SearchRSS Plugin by WebGroup Media LLC - http://www.webgroupmedia.com/</generator>
<docs>http://blogs.law.harvard.edu/tech/rss</docs>
<?php
if($db->num_rows($res)) {
while($row = $db->fetch_array($res)) {
$pubDate = date('r', $row['dateline']);
$baseurl = WGMRSS_BASEURL;
$title = utf8_encode($row['title']);
$text = utf8_encode($row['pagetext_html']);
echo <<< HERE
<item>
<title><![CDATA[${title}]]></title>
<author>${row['username']}</author>
<link>${baseurl}showthread.php?p=${row['postid']}</link>
<guid>${row['postid']}</guid>
<description><![CDATA[${text}]]></description>
<content:encoded><![CDATA[]]></content:encoded>
<pubDate>${pubDate}</pubDate>
</item>
HERE;
}
}
?>
</channel>
</rss>
<?php
exit;
}
darren
01-04-2008, 09:12 PM
Jeff, is this working with the latest 3.6.8 PL2 vBulletin?
We're getting a blank feed when trying this code. Looks like the db query isn't returning a result. We've got fulltext as the search option in the vb admincp.
jstanden
01-09-2008, 02:00 AM
Hey Darren!
I can take a look at -PL2 when I get a comment moments.
In the meantime, can you post your output from the following SQL statements?
describe post;
describe postparsed;
describe thread;
Thanks!
darren
01-15-2008, 09:12 PM
post
Field Type Null Key Default Extra
postid int(10) unsigned PRI NULL auto_increment
threadid int(10) unsigned MUL 0
parentid int(10) unsigned 0
username varchar(100)
userid int(10) unsigned MUL 0
title varchar(250) MUL
dateline int(10) unsigned 0
pagetext mediumtext
allowsmilie smallint(6) 0
showsignature smallint(6) 0
ipaddress varchar(15)
iconid smallint(5) unsigned 0
visible smallint(6) 0
attach smallint(5) unsigned 0
infraction smallint(5) unsigned 0
reportthreadid int(10) unsigned 0
postparsed
Field Type Null Key Default Extra
postid int(10) unsigned PRI 0
dateline int(10) unsigned MUL 0
hasimages smallint(6) 0
pagetext_html mediumtext
styleid smallint(5) unsigned PRI 0
languageid smallint(5) unsigned PRI 0
thread
Field Type Null Key Default Extra
threadid int(10) unsigned PRI NULL auto_increment
title varchar(250) MUL
firstpostid int(10) unsigned 0
lastpost int(10) unsigned MUL 0
forumid smallint(5) unsigned MUL 0
pollid int(10) unsigned MUL 0
open smallint(6) 0
replycount int(10) unsigned 0
postusername varchar(100)
postuserid int(10) unsigned MUL 0
lastposter varchar(50)
dateline int(10) unsigned MUL 0
views int(10) unsigned 0
iconid smallint(5) unsigned 0
notes varchar(250)
visible smallint(6) 0
sticky smallint(6) 0
votenum smallint(5) unsigned 0
votetotal smallint(5) unsigned 0
attach smallint(5) unsigned 0
similar varchar(55)
hiddencount int(10) unsigned 0
deletedcount smallint(5) unsigned 0
lastpostid int(10) unsigned 0
Hildy
01-24-2008, 07:59 PM
Hm. I see nothing in the structure that would indicate a problem. Try running the search manually in a mysql window and see if it gives any errorsSELECT p.postid, t.title, p.dateline, p.username,
pp.pagetext_html, MATCH(p.title,pagetext) AGAINST ('%s') AS score
FROM post p
INNER JOIN thread t ON (t.threadid=p.threadid)
INNER JOIN postparsed pp ON (pp.postid=p.postid)
WHERE MATCH(p.title,pagetext) AGAINST ('%s')
GROUP BY p.threadid
ORDER BY score DESC
LIMIT 0,100;
darren
01-24-2008, 10:05 PM
From phpmyadmin, that returns
MySQL returned an empty result set (i.e. zero rows). (Query took 0.0004 sec)
Hildy
01-24-2008, 10:42 PM
Sounds like it's running correctly.... Silly question: you *did* replace both '%s's with a search string in your manual attempt, yes?
Also, what do you get from running the same statement, but without the WHERE clause? (You may want to change the limit to 10, too)
I'll have to do some research on the "MATCH ... AGAINST" sql...
darren
01-25-2008, 04:52 PM
I didn't replace it, but even if I do, it still the same result. Zero rows.
If I remove the WHERE clause, it still gives zero rows too.
Hildy
01-25-2008, 09:38 PM
Any chance I could get a login to take a look at it?
darren
01-25-2008, 10:23 PM
Hildy, I've sent you an email re this. Thanks.
Hildy
01-29-2008, 07:01 PM
Ha! For some reason, your `postparsed` table is empty (possibly a config issue where vBulletin isn't using that table?). At any rate, update the sql in the F&R adapter to:SELECT p.postid, t.title, p.dateline, p.username,
p.pagetext, MATCH(p.title,pagetext) AGAINST ('%s') AS score
FROM post p
INNER JOIN thread t ON (t.threadid=p.threadid)
WHERE MATCH(p.title,pagetext) AGAINST ('%s')
GROUP BY p.threadid
ORDER BY score DESC
LIMIT 0,100;
That sql got me results on your server, so it should work in the adapter. :-)
darren
01-29-2008, 07:31 PM
Great. Not sure why our postparsed table would be empty - don't recall changing any settings for this.
Anyway, the updated SQL works perfectly :)
Thank you.
This gives us...
Parse error: syntax error, unexpected $end in (directory removed for security reasons)/searchrss.php on line 57
ahh you guys got some php5 syntax in here? Mind most people are going to be running their vb on php4. :P (Edit: That's just a guess, couldn't find any version requirement on thier site).
weird, ok I fixed a missing bracket and changed the <<< code, then had to enable full text searching in VBulletin. Now generating blank rss results. Updated the SQL with Hildy's new code and working a treat :) Very sexy, need to update your instructions for this though to make it a little easier for noobs! =)
(Edit: Missing bracket was due to my rather excellent copy and paste skills)
Hildy
02-19-2008, 05:28 PM
Glad to hear you got it working, trax, and thanks for posting why. :-)
vBulletin® v3.7.2, Copyright ©2000-2008, Jelsoft Enterprises Ltd.