Jacob's TN Searchy Thingy

Make suggestions and report problems.
User avatar
JacobH
Quinta do Vesuvio 1994
Posts: 3300
Joined: 16:37 Sat 03 May 2008
Location: London, UK
Contact:

Jacob's TN Searchy Thingy

Post by JacobH »

Admin2 Note: This thread split from here.

I don’t know if this would be helpful, or not, but I’ve been working on a similar script that I have been using to search through the TN on :ftlop:. I’ve got a copy up here: http://www.jacob-head.com/ftlop/ . It’s password protected, because I still am not quite sure whether Roy and the rest of the FTLOP team are happy with me publicising it, but feel free to try it. The username is: theportforum and the password: NovalNacional.

It can be updated automatically (though at the moment this is inactive so I don’t overuse the :ftlop: servers), which may be an advantage. It’s main disadvantages are that currently it doesn’t have names and dates for each note and it just gives the topic subject, rather than refining it.

I’m not sure whether there is much need for something like that. I’ve occasionally found it useful to use it to search for (say) all the VPs from one year, but perhaps for most uses the basic list will suffice!
User avatar
DRT
Fonseca 1966
Posts: 15779
Joined: 23:51 Wed 20 Jun 2007
Location: Chesterfield, UK
Contact:

Post by DRT »

Jacob,

Is this scanning the :ftlop: users TNs or Roy's own TNs? If it's the former then it seems it is not really much different to a BOT scanning for keywords on any public site. If it's the latter I would imagine Roy will have a major issue with it, especially if you have subscribed to :ftlop: to gain access to the data.

Derek
"The first duty of Port is to be red"
Ernest H. Cockburn
User avatar
DRT
Fonseca 1966
Posts: 15779
Joined: 23:51 Wed 20 Jun 2007
Location: Chesterfield, UK
Contact:

Post by DRT »

I just answered my own question by trying it and finding one of my own TNs!

Have you contacted Roy or Stewart about it? If not I would suggest you do to see what they think.

Derek
"The first duty of Port is to be red"
Ernest H. Cockburn
User avatar
KillerB
Taylor Quinta de Vargellas 1987
Posts: 2425
Joined: 22:09 Wed 20 Jun 2007
Location: Sky Blue City, England

Post by KillerB »

I like that - it's different from what we do but useful nonetheless.
Port is basically a red drink
User avatar
JacobH
Quinta do Vesuvio 1994
Posts: 3300
Joined: 16:37 Sat 03 May 2008
Location: London, UK
Contact:

Post by JacobH »

I did, but it was at about a month ago; the same time they were relaunching the site so didn’t get a chance to take it anywhere.

Although, as you say, it affects their site no more than Google, I think it would be a bit rude to advertise it on there without their permission!
Conky
Fonseca 1980
Posts: 1770
Joined: 23:51 Wed 20 Jun 2007

Post by Conky »

A search facility does seem good. To those with technical know how, is it clearly better than the basic search on the green toolbar?
User avatar
DRT
Fonseca 1966
Posts: 15779
Joined: 23:51 Wed 20 Jun 2007
Location: Chesterfield, UK
Contact:

Post by DRT »

Jacob,

Would it take lots of effort to create a version of your index/search facility that points at the TNs on :tpf:?

Obviously it would be good for The :tpf: Collective to agree to allow that to happen but I would be interested to know how easy t would be to make possible.

Derek
"The first duty of Port is to be red"
Ernest H. Cockburn
User avatar
JacobH
Quinta do Vesuvio 1994
Posts: 3300
Joined: 16:37 Sat 03 May 2008
Location: London, UK
Contact:

Post by JacobH »

No, it shouldn’t be too difficult…To produce a manually-updated version on my site (like the :ftlop: one) would, touch wood, only take about 10 minutes…Adding names and dates should, I hope, also be quite straight-forward.

Getting it to automatically update would be a bit more complex as it would have to be hosted on the :tpf: server, reading the threads off the forum database. I think that would be a Sunday-afternoon project!
User avatar
DRT
Fonseca 1966
Posts: 15779
Joined: 23:51 Wed 20 Jun 2007
Location: Chesterfield, UK
Contact:

Post by DRT »

JacobH wrote:No, it shouldn’t be too difficult…To produce a manually-updated version on my site (like the :ftlop: one) would, touch wood, only take about 10 minutes…Adding names and dates should, I hope, also be quite straight-forward.

Getting it to automatically update would be a bit more complex as it would have to be hosted on the :tpf: server, reading the threads off the forum database. I think that would be a Sunday-afternoon project!
Interesting! Perhaps others could chip in with thoughts?

Derek
"The first duty of Port is to be red"
Ernest H. Cockburn
User avatar
KillerB
Taylor Quinta de Vargellas 1987
Posts: 2425
Joined: 22:09 Wed 20 Jun 2007
Location: Sky Blue City, England

Post by KillerB »

If you show me how it's done then we will give it a go. I can write SQL queries that will achieve the same thing so see no reason not to achieve it with whatever else we've got.
Port is basically a red drink
User avatar
JacobH
Quinta do Vesuvio 1994
Posts: 3300
Joined: 16:37 Sat 03 May 2008
Location: London, UK
Contact:

Post by JacobH »

I made the necessary changes and put a version for :tpf: on site at http://www.jacob-head.com/tpf. For the record, the necessary changes did only take 10 minutes, though working out that the reason that the same 10 bottles were shown 15 times each was not because of a bug in the code but because I’d downloaded the same page 15 times, took considerably more… :opps:

It’s a bit rough-and-ready but hopefully shows what might be possible.

KillerB, the only non-trivial bit of code is this one which takes an arbitrary string and then returns an array containing a guess as to the shipper, type of port and vintage referred to in that string.

The index is generated in three stages, using XML. First we get the raw data, consisting of topic names and urls, and format them into an XML file as follows:

Code: Select all

<forum>
<topic>
<name>Cruz Ruby</name>
<url>http://www.xyz.com/xzy.php</url>
</topic>
</forum>
For these mock-ups, I‘ve been doing this by grabbing the forum index source code with wget and parsing it through some sed scripts.

Stage two is this set of code, which loops through the XML and produces:

Code: Select all

<forum>
<topic>
<name>Cruz Ruby</name>
<type>Ruby</type>
<year>nv</year>
<url>http://www.xyz.com/xzy.php</url>
</topic>
</forum>
Finally, stage three is the index.php (source code) which just takes the XML and displays/manipulates it.

With database access, stage one would be eliminated and I could incorporate stage 2 into the index.php file in a slightly more sophisticated way (so as to make it auto-update).

Alternatively, perhaps there might be some use for the basic portid function in generating the current static list, so you don’t have to do any manual sorting?
User avatar
KillerB
Taylor Quinta de Vargellas 1987
Posts: 2425
Joined: 22:09 Wed 20 Jun 2007
Location: Sky Blue City, England

Post by KillerB »

Thank you Jacob. We will look at ways of getting this directly on the TPF site with access to the Database. At the moment, I need sleep.
Port is basically a red drink
User avatar
JacobH
Quinta do Vesuvio 1994
Posts: 3300
Joined: 16:37 Sat 03 May 2008
Location: London, UK
Contact:

Post by JacobH »

I've finally had a chance to patch together some pseudo-php which would be how I would automate the script with database access if such a system might be helpful.

The passages marked **SQL** would, of course, need to be converted into real SQL calls, but as that's very much dependant on your set-up, it seemed easiest to leave them in pseudo-form. It uses my PortID function to do the hard work. I'm just about finishing a version 1.01 which will split crusted, garrafeira and white out of "ruby &c." and do some sub-types (e.g. identify if it is a 10/20/30/40 year-old tawny; ruby or ruby reserve; which sort of white and have a guess at the quinta for sqvp).

Code: Select all

include('./portid.inc');

##We only need to update if there have been new topics.
##How many topics are there now?

**SQL**
$currenttopics = SELECT forum_topics FROM phpbb_forums WHERE forum_id = 2;

##How many topics where there last time?

$oldtopics=file_get_contents('numberoftopics.txt');

##If there are now more topics than last time, then we rebuild the database
if($currenttopics > $oldtopics){


##Create new XML document using the DOM
$xml = new DomDocument('1.0');

##Build an array of topic numbers from the TN forum

$arrayoftopics=array();

**SQL**
$arrayoftopics = SELECT topic_id FROM phpbb_topics WHERE forum_id = 2;

##Loop through the array

foreach($arrayoftopics as $currenttopic){

        $newtopic=$xml->addChild('topic');

        **SQL**
        $topicname = SELECT topic_title FROM php_topics WHERE topic_id =
$currenttopic;

        $newtopic->addChild('name', $topicname);

        $newtopic->addChild('url',
'http://www.theportforum.com/viewtopic.php?t='.$currentopic );

        $portid=portid($topicname);

        $newtopic->addChild('year', $portid[0]);
        $newtopic->addChild('producer', $portid[1]);
        $newtopic->addChild('type', $portid[2]);

        **SQL**
        $topictime = SELECT topic_time FROM php_topics WHERE topic_id =
$currenttopic;

        $newtopic->addChild('time', $topictime);

        **SQL**
        $topic_poster = SELECT topic_poster FROM php_topics WHERE topic_id = $currenttopic;

        $newtopic->addChild('author', $topic_poster);

}

file_put_contents('topics.xml',$xml->asXML());
}

***here continues the index.php file from previous post to display the database***
User avatar
benread
Niepoort 1977
Posts: 1555
Joined: 21:36 Thu 17 Apr 2008
Location: Reigate, Surrey
Contact:

Post by benread »

Jacob,

Would this search all tasting notes or just those already indexed in the "A-Z"?
Ben
-------
Vintage 1970 and now proud owner of my first ever 'half-century'!
User avatar
KillerB
Taylor Quinta de Vargellas 1987
Posts: 2425
Joined: 22:09 Wed 20 Jun 2007
Location: Sky Blue City, England

Post by KillerB »

Jacob,

The SQL is pretty accurate. I do SQL, I don't do php, Treacle does a bit of php but doesn't do SQL :roll:

Let me know what you need to implement it and we will see what we can get done,

Cheers,

Alex
Port is basically a red drink
User avatar
JacobH
Quinta do Vesuvio 1994
Posts: 3300
Joined: 16:37 Sat 03 May 2008
Location: London, UK
Contact:

Post by JacobH »

Ben, yes, the idea is that it will automatically index the posts as they are posted. There are a few difficulties in doing it automatically but thankfully as the number of Port shippers is quite low (120 or so), its not too hard to put in individual exceptions, even if we do have more varieties of wine than other regions.

Alex, that’s a shame, though completely understandable as getting php and SQL to co-operate nicely is complete pain! I can do a bit of php and a bit of SQL at the terminal, but hate doing them together.

What might be useful to know is what database you are running as the php commands are completely different for PostgreSQL and MySQL. It would also be helpful to know if those SQL calls will work on the set-up here (e.g. with table names etc.) or if I’ve made any stupid mistakes with them (it’s a while since I’ve SQLed in anger).

I’ll try to finish off the new version of PortID and sort out the index page so it’s a bit more efficient. It’ll then just be a case of stringing the lot together, uploading it and seeing if it works :D
User avatar
DRT
Fonseca 1966
Posts: 15779
Joined: 23:51 Wed 20 Jun 2007
Location: Chesterfield, UK
Contact:

Post by DRT »

This thread has taken on a new meaning and I would suggest that "Jacob's TN searchy thing" deserves it's own thread so that this one can return to being a prompt to KillerB and Jdaw to get moving on updating the TN Index.

Any objections to splitting from the point where Jacob first mentioned his searchy thing?

Derek
"The first duty of Port is to be red"
Ernest H. Cockburn
User avatar
JacobH
Quinta do Vesuvio 1994
Posts: 3300
Joined: 16:37 Sat 03 May 2008
Location: London, UK
Contact:

Post by JacobH »

That makes sense to me. Fire away! Image
User avatar
DRT
Fonseca 1966
Posts: 15779
Joined: 23:51 Wed 20 Jun 2007
Location: Chesterfield, UK
Contact:

Post by DRT »

JacobH wrote:That makes sense to me. Fire away! Image
Done. :wink:
"The first duty of Port is to be red"
Ernest H. Cockburn
User avatar
JacobH
Quinta do Vesuvio 1994
Posts: 3300
Joined: 16:37 Sat 03 May 2008
Location: London, UK
Contact:

Post by JacobH »

Although the logs suggest no-one was using it, just to let people know, I have deleted the user account for the :FTLOP: notes, as I have been informed that my posting it here was, unfortunately, causing a certain amount of annoyance at :FTLOP:.
Conky
Fonseca 1980
Posts: 1770
Joined: 23:51 Wed 20 Jun 2007

Post by Conky »

JacobH wrote:Although the logs suggest no-one was using it, just to let people know, I have deleted the user account for the :FTLOP: notes, as I have been informed that my posting it here was, unfortunately, causing a certain amount of annoyance at :FTLOP:.
It does... :roll:

But dont worry, 99% of us are all friends. :D
Roy Hersh
Niepoort LBV
Posts: 283
Joined: 21:55 Mon 31 Dec 2007

Post by Roy Hersh »

Thank you Jacob! No harm, no foul, as it was more of an "ethics" issue for me, not one with TPF. But that is firmly behind us now.

As to the other 1%, Alan alludes to ... he may actually someday fill that void, once we meet in person and I am assured his service revolver is not loaded. I am still unsure of his ability to handle copious amounts of VP in one sitting. 88) But that is what offlines are for, to get to know folks in person where on line personas may not be as endearing. I am sure that of course, works both ways. :D
Conky
Fonseca 1980
Posts: 1770
Joined: 23:51 Wed 20 Jun 2007

Post by Conky »

Fair comment Roy,

and sadly I would be at a grave disadvantage if Andy was there. We do not carry guns as a matter of course. I have other methods...but none of them out run a speeding bullet. :D
User avatar
Alex Bridgeman
Graham’s 1948
Posts: 14880
Joined: 13:41 Mon 25 Jun 2007
Location: Berkshire, UK

Post by Alex Bridgeman »

Roy

One of the weaknesses of FTLOP at present is that there is a great wealth of tasting notes from the forum members, but no index or easy way to find a note for a particular port (this is the only reason that I do not post tasting notes on FTLOP very often - once I post them it is almost impossible to refer back to them after a few days have gone by). Jacob's tasting note search string could help you to fill this gap relatively quickly...

Alex
Top Ports in 2023: Taylor 1896 Colheita, b. 2021. A perfect Port.

2024: Niepoort 1900 Colheita, b.1971. A near perfect Port.
Conky
Fonseca 1980
Posts: 1770
Joined: 23:51 Wed 20 Jun 2007

Post by Conky »

For a Computer half-wit like myself, what stage are we up to? Any estimated time-scale, or is it just when a few of you get round to it (Which I completely understand)
Post Reply