[Tech] meta category

[Florent Daignière (NextGen$)]

* Dave Baker <dbkr at freenetproject.org> [2006-11-25 22:02:58]:

> On Saturday 25 November 2006 21:48, sich wrote:
> > toad a écrit :
> > > Why do you need it to survive the HTML filter? Is there some benefit to
> > > fetching stuff through fproxy?
> > 
> > I currently use mnogosearch as spider to find new freesite, and he use 
> > fproxy (I knon it's not the best way, but I am a poor devl).
> 
> If you're not that confident with development, why not use one of the plethora 
> of existing Freenet spiders rather than writing yet *another* one?

Well, when he started writing his spider, Librarian didn't exist ;)

> 
> > If I can find this meta in the site this will be easier ton sort the 
> > sites on the index.
> > I currently fetch :
> >   - Title
> >   - Content-Language
> >   - Description
> > 
> > When I have find the sites with mnogosearch, I use some php scripts that 
> > I have write (they use FCP), to test if the site is responding, if we 
> > have the last version, etc.
> > 
> > 

Toad is right here ... your spider ought to use FCP and not fproxy and
its content-filter.

-- 
NextGen$. 
"On peut obéïr aux lois en souhaitant qu'elles changent, comme on sert à la guerre en souhaitant la paix."
Merleau Ponty - L'éloge de la philosophie
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://emu.freenetproject.org/pipermail/tech/attachments/20061126/25bba61c/attachment.pgp