From dbkr at freenetproject.org  Thu May  1 08:19:45 2008
From: dbkr at freenetproject.org (dbkr at freenetproject.org)
Date: Thu,  1 May 2008 08:19:45 +0000 (UTC)
Subject: [Freemail] r19642 - trunk/apps/Freemail/src/freemail
Message-ID: <20080501081945.08AF747AEEF@freenetproject.org>

Author: dbkr
Date: 2008-05-01 08:19:44 +0000 (Thu, 01 May 2008)
New Revision: 19642

Modified:
   trunk/apps/Freemail/src/freemail/OutboundContact.java
Log:
Muppet! Lowest UID, not highest - not that it worked anyway.


Modified: trunk/apps/Freemail/src/freemail/OutboundContact.java
===================================================================
--- trunk/apps/Freemail/src/freemail/OutboundContact.java	2008-05-01 05:27:12 UTC (rev 19641)
+++ trunk/apps/Freemail/src/freemail/OutboundContact.java	2008-05-01 08:19:44 UTC (rev 19642)
@@ -284,32 +284,39 @@
 	 */
 	private String getCurrentLowestSlot() {
 		QueuedMessage[] queue = getSendQueue();
-		if (queue.length > 0) {
-			int messageWithHighestUid = 0;
-			for (int i = 0; i < queue.length; i++) {
-				if (queue[i] == null) continue;
-				if (queue[i].uid > messageWithHighestUid) messageWithHighestUid = i;
+		int messageWithLowestUid = 0;
+		int lowestUid = Integer.MAX_VALUE;
+		// queue.length == 0 doesn't necessarily imply there's anything
+		// in the queue - the array can contain null values (due to the
+		// sucky way in which getSendQueue() works by returning an array)
+		for (int i = 0; i < queue.length; i++) {
+			if (queue[i] == null) continue;
+			if (queue[i].uid < lowestUid) {
+				messageWithLowestUid = i;
+				lowestUid = queue[i].uid;
 			}
-			return queue[messageWithHighestUid].slot;
+		}
+		if (lowestUid < Integer.MAX_VALUE) return queue[messageWithLowestUid].slot;
+
+		// No messages in the queue, so the current lowest slot is the
+		// next slot we'll insert a message to.
+		String retval = this.contactfile.get("nextslot");
+		if (retval != null) {
+			return retval;
 		} else {
-			String retval = this.contactfile.get("nextslot");
-			if (retval != null) {
-				return retval;
-			} else {
-				Logger.minor(this, "Generating first slot for contact");
-				SecureRandom rnd = new SecureRandom();
-				SHA256Digest sha256 = new SHA256Digest();
-				byte[] buf = new byte[sha256.getDigestSize()];
-				
-				rnd.nextBytes(buf);
-				
-				String firstSlot = Base32.encode(buf);
-				
-				this.contactfile.put("nextslot", Base32.encode(buf));
-				
-				return firstSlot;
+			Logger.minor(this, "Generating first slot for contact");
+			SecureRandom rnd = new SecureRandom();
+			SHA256Digest sha256 = new SHA256Digest();
+			byte[] buf = new byte[sha256.getDigestSize()];
+			
+			rnd.nextBytes(buf);
+			
+			String firstSlot = Base32.encode(buf);
+			
+			this.contactfile.put("nextslot", Base32.encode(buf));
+			
+			return firstSlot;
 			}
-		}
 	}
 	
 	private byte[] getAESParams() {



From alexlehm at myrealbox.com  Thu May  1 15:01:47 2008
From: alexlehm at myrealbox.com (Alexander Lehmann)
Date: Thu, 01 May 2008 17:01:47 +0200
Subject: [Freemail] Log: Don't re-insert mail to different slots
Message-ID: <1209654107.c7f0cb3calexlehm@myrealbox.com>

I think it may be useful to first try to insert the message to a new slot (in case the user has missed a slot somehow) and later to retry rts or is this a situation that doesn't seem likely.

freemail-request at freenetproject.org wrote:
> Modified:
>    trunk/apps/Freemail/src/freemail/OutboundContact.java
> Log:
> Don't re-insert mail to different slots - re-send the RTS instead. This will allow contacts to be re-established if the inbound contact has been deleted.
> 


-- 
Alexander Lehmann <alexlehm at myrealbox.com>




From dbkr at freenetproject.org  Thu May  1 15:23:59 2008
From: dbkr at freenetproject.org (Dave Baker)
Date: Thu, 1 May 2008 16:23:59 +0100
Subject: [Freemail] Log: Don't re-insert mail to different slots
In-Reply-To: <1209654107.c7f0cb3calexlehm@myrealbox.com>
References: <1209654107.c7f0cb3calexlehm@myrealbox.com>
Message-ID: <200805011623.59719.dbkr@freenetproject.org>

On Thursday 01 May 2008 16:01:47 Alexander Lehmann wrote:
> I think it may be useful to first try to insert the message to a new slot
> (in case the user has missed a slot somehow) and later to retry rts or is
> this a situation that doesn't seem likely.
>
> freemail-request at freenetproject.org wrote:
> > Modified:
> >    trunk/apps/Freemail/src/freemail/OutboundContact.java
> > Log:
> > Don't re-insert mail to different slots - re-send the RTS instead. This
> > will allow contacts to be re-established if the inbound contact has been
> > deleted.

Well, I was thinking along the lines that the user missing a slot shouldn't 
happen, since we'll only advance through a slot if we find a message on it. I 
don't think it's every likley to be that a user cann't fetch one ket but can 
fetch the same data on a different key.

Feedback welcome though.


Dave


From dbkr at freenetproject.org  Thu May  1 22:28:22 2008
From: dbkr at freenetproject.org (dbkr at freenetproject.org)
Date: Thu,  1 May 2008 22:28:22 +0000 (UTC)
Subject: [Freemail] r19667 - trunk/apps/Freemail/src/freemail
Message-ID: <20080501222822.10434479906@freenetproject.org>

Author: dbkr
Date: 2008-05-01 22:28:21 +0000 (Thu, 01 May 2008)
New Revision: 19667

Modified:
   trunk/apps/Freemail/src/freemail/RTSFetcher.java
Log:
Don't count a poll if we encountered network errors


Modified: trunk/apps/Freemail/src/freemail/RTSFetcher.java
===================================================================
--- trunk/apps/Freemail/src/freemail/RTSFetcher.java	2008-05-01 22:28:06 UTC (rev 19666)
+++ trunk/apps/Freemail/src/freemail/RTSFetcher.java	2008-05-01 22:28:21 UTC (rev 19667)
@@ -117,10 +117,10 @@
 		for (i = 1 - MAX_DAYS_BACK; i <= 0; i++) {
 			String datestr = DateStringFactory.getOffsetKeyString(i);
 			if (log.getPasses(datestr) < PASSES_PER_DAY) {
-				this.fetch_day(log, datestr);
+				boolean successfulPoll = this.fetch_day(log, datestr);
 				// don't count passes for today since more
 				// mail may arrive
-				if (i < 0) {
+				if (i < 0 && successfulPoll) {
 					log.incPasses(datestr);
 				}
 			}
@@ -139,7 +139,11 @@
 		String date;
 	}
 	
-	private void fetch_day(RTSLog log, String date) throws ConnectionTerminatedException {
+	/**
+	 * @return true if the day was sucessfully polled, false if there were network-type errors and the polling shouldn't count
+	 *              as a valid check of that day's slots.
+	 */
+	private boolean fetch_day(RTSLog log, String date) throws ConnectionTerminatedException {
 		HighLevelFCPClient fcpcli;
 		fcpcli = new HighLevelFCPClient();
 		
@@ -155,6 +159,7 @@
 		sm.setPollAhead(POLL_AHEAD);
 		
 		int slot;
+		boolean success = true;
 		while ( (slot = sm.getNextSlotNat()) > 0) {
 			Logger.minor(this,"trying to fetch "+keybase+slot);
 			
@@ -184,11 +189,16 @@
 					sm.incPollAhead();
 				} else if (fe.getCode() == FCPFetchException.DATA_NOT_FOUND) {
 					Logger.minor(this,keybase+slot+": no RTS.");
+				} else if (fe.isNetworkError()) {
+					// Freenet is having special moment. This doesn't count as a valid poll.
+					success = false;
 				} else {
-					Logger.minor(this,keybase+slot+": other non-fatal fetch error:"+fe.getMessage());
+					// We've covered most things above, so I think this should a fairly exceptional case. Let's log it at error.
+					Logger.error(this,keybase+slot+": other non-fatal fetch error:"+fe.getMessage());
 				}
 			}
 		}
+		return success;
 	}
 	
 	public void saveSlots(String slots, Object userdata) {



From dbkr at freenetproject.org  Thu May  1 22:56:16 2008
From: dbkr at freenetproject.org (dbkr at freenetproject.org)
Date: Thu,  1 May 2008 22:56:16 +0000 (UTC)
Subject: [Freemail] r19668 - trunk/apps/Freemail/src/freemail
Message-ID: <20080501225616.780B4479A1E@freenetproject.org>

Author: dbkr
Date: 2008-05-01 22:56:16 +0000 (Thu, 01 May 2008)
New Revision: 19668

Modified:
   trunk/apps/Freemail/src/freemail/OutboundContact.java
Log:
D'oh - clearly that will happen the first time a contact is used.


Modified: trunk/apps/Freemail/src/freemail/OutboundContact.java
===================================================================
--- trunk/apps/Freemail/src/freemail/OutboundContact.java	2008-05-01 22:28:21 UTC (rev 19667)
+++ trunk/apps/Freemail/src/freemail/OutboundContact.java	2008-05-01 22:56:16 UTC (rev 19668)
@@ -304,21 +304,25 @@
 		if (retval != null) {
 			return retval;
 		} else {
-			Logger.minor(this, "Generating first slot for contact");
-			SecureRandom rnd = new SecureRandom();
-			SHA256Digest sha256 = new SHA256Digest();
-			byte[] buf = new byte[sha256.getDigestSize()];
-			
-			rnd.nextBytes(buf);
-			
-			String firstSlot = Base32.encode(buf);
-			
-			this.contactfile.put("nextslot", Base32.encode(buf));
-			
-			return firstSlot;
-			}
+			return generateFirstSlot();
+		}
 	}
 	
+	private String generateFirstSlot() {
+		Logger.minor(this, "Generating first slot for contact");
+		SecureRandom rnd = new SecureRandom();
+		SHA256Digest sha256 = new SHA256Digest();
+		byte[] buf = new byte[sha256.getDigestSize()];
+		
+		rnd.nextBytes(buf);
+		
+		String firstSlot = Base32.encode(buf);
+		
+		this.contactfile.put("nextslot", Base32.encode(buf));
+		
+		return firstSlot;
+	}
+	
 	private byte[] getAESParams() {
 		String params = this.contactfile.get("aesparams");
 		if (params != null) {
@@ -562,8 +566,7 @@
 	private String popNextSlot() {
 		String slot = this.contactfile.get("nextslot");
 		if (slot == null) {
-			Logger.error(this, "Contact has no 'nextslot' prop! This shouldn't happen!");
-			return null;
+			return generateFirstSlot();
 		}
 		SHA256Digest sha256 = new SHA256Digest();
 		sha256.update(Base32.decode(slot), 0, Base32.decode(slot).length);



From alexlehm at myrealbox.com  Sat May  3 14:01:30 2008
From: alexlehm at myrealbox.com (Alexander Lehmann)
Date: Sat, 03 May 2008 16:01:30 +0200
Subject: [Freemail] Log: Don't re-insert mail to different slots
Message-ID: <1209823290.c8047d7calexlehm@myrealbox.com>

I think the rationale behind polling ahead on the hash slots was that the user may miss an inserted key sometimes, if this is the case, it would be useful to insert the message to a new slot when starting a new rts, it that cannot happen, it is not necessary since the old key can be used.

However if we do not expect this to happen, do we really need to poll ahead on the keys?

> Message: 2
> Date: Thu, 1 May 2008 16:23:59 +0100
> From: Dave Baker <dbkr at freenetproject.org>
> Subject: Re: [Freemail] Log: Don't re-insert mail to different slots
> To: freemail at freenetproject.org
> Message-ID: <200805011623.59719.dbkr at freenetproject.org>
> Content-Type: text/plain;  charset="iso-8859-1"
> 
> On Thursday 01 May 2008 16:01:47 Alexander Lehmann wrote:
> > I think it may be useful to first try to insert the message to a new slot
> > (in case the user has missed a slot somehow) and later to retry rts or is
> > this a situation that doesn't seem likely.
> >
> > freemail-request at freenetproject.org wrote:
> > > Modified:
> > >    trunk/apps/Freemail/src/freemail/OutboundContact.java
> > > Log:
> > > Don't re-insert mail to different slots - re-send the RTS instead. This
> > > will allow contacts to be re-established if the inbound contact has been
> > > deleted.
> 
> Well, I was thinking along the lines that the user missing a slot shouldn't 
> happen, since we'll only advance through a slot if we find a message on it. I 
> don't think it's every likley to be that a user cann't fetch one ket but can 
> fetch the same data on a different key.
> 
> Feedback welcome though.
> 
> 
> Dave


-- 
Alexander Lehmann <alexlehm at myrealbox.com>




From dbkr at freenetproject.org  Mon May  5 14:39:37 2008
From: dbkr at freenetproject.org (dbkr at freenetproject.org)
Date: Mon,  5 May 2008 14:39:37 +0000 (UTC)
Subject: [Freemail] r19766 - trunk/apps/Freemail/src/freemail
Message-ID: <20080505143937.5656847815C@freenetproject.org>

Author: dbkr
Date: 2008-05-05 14:39:37 +0000 (Mon, 05 May 2008)
New Revision: 19766

Modified:
   trunk/apps/Freemail/src/freemail/AccountManager.java
   trunk/apps/Freemail/src/freemail/Freemail.java
Log:
Release 0.1-11 and update the welcome message to point to the FMS board and the new Freesite.


Modified: trunk/apps/Freemail/src/freemail/AccountManager.java
===================================================================
--- trunk/apps/Freemail/src/freemail/AccountManager.java	2008-05-05 13:59:31 UTC (rev 19765)
+++ trunk/apps/Freemail/src/freemail/AccountManager.java	2008-05-05 14:39:37 UTC (rev 19766)
@@ -399,14 +399,16 @@
 		ps.println("");
 		ps.println(to);
 		ps.println("");
-		ps.println("But you'll probably want a shorter one. To do this, run Freemail with the --shortaddress argument, followed by your account name and the part you'd like before the '.freemail'. For example:");
+		ps.println("There are several places to discuss Freemail, report problems, or ask for help:");
+		ps.println(" * The 'freemail' board on FMS");
+		ps.println(" * The bug tracker: https://bugs.freenetproject.org/ (select 'Freemail' in the top right).");
+		ps.println(" * #freemail on irc.freenode.net");
+		ps.println(" * The mailing list at http://emu.freenetproject.org/cgi-bin/mailman/listinfo/freemail.");
 		ps.println("");
-		ps.println("java -jar freemail.jar --shortaddress bob bobshouse");
+		ps.println("Don't forget to stay up to date with the Freemail news and latest version at the freesite, which can be found at:");
 		ps.println("");
-		ps.println("Try to pick something unique - Freemail will tell you if somebody has already taken the address you want. These short addresses are *probably* secure, but not absolutely. If you want to be sure, use the long address.");
+		ps.println("USK at xOg49GNltumTJJzj0fVzuGDpo4hJUsy2UsGQkjE7NY4,EtUH5b9gGpp8JiY-Bm-Y9kHX1q-yDjD-9oRzXn21O9k,AQACAAE/freemail/-1/");
 		ps.println("");
-		ps.println("If you find a bug, or would like something changed in Freemail, visit our bug tracker at https://bugs.freenetproject.org/ (and select 'Freemail' in the top right). You can also drop into #freemail on irc.freenode.net to discuss, or sign up to the mailing list at http://emu.freenetproject.org/cgi-bin/mailman/listinfo/freemail.");
-		ps.println("");
 		ps.println("Happy Freemailing!");
 		ps.println("");
 		ps.println("");

Modified: trunk/apps/Freemail/src/freemail/Freemail.java
===================================================================
--- trunk/apps/Freemail/src/freemail/Freemail.java	2008-05-05 13:59:31 UTC (rev 19765)
+++ trunk/apps/Freemail/src/freemail/Freemail.java	2008-05-05 14:39:37 UTC (rev 19766)
@@ -37,7 +37,7 @@
 public abstract class Freemail implements ConfigClient {
 	public static final int VER_MAJOR = 0;
 	public static final int VER_MINOR = 1;
-	public static final int BUILD_NO = 10;
+	public static final int BUILD_NO = 11;
 	public static final String VERSION_TAG = "Pet Shop";
 
 	private static final String TEMPDIRNAME = "temp";



From dbkr at freenetproject.org  Wed May 14 22:00:44 2008
From: dbkr at freenetproject.org (dbkr at freenetproject.org)
Date: Wed, 14 May 2008 22:00:44 +0000 (UTC)
Subject: [Freemail] r19933 - trunk/apps/Freemail/src/freemail/utils
Message-ID: <20080514220044.6687247902A@freenetproject.org>

Author: dbkr
Date: 2008-05-14 22:00:44 +0000 (Wed, 14 May 2008)
New Revision: 19933

Modified:
   trunk/apps/Freemail/src/freemail/utils/EmailAddress.java
Log:
Just make bits of email addresses lowercase - not the real name.


Modified: trunk/apps/Freemail/src/freemail/utils/EmailAddress.java
===================================================================
--- trunk/apps/Freemail/src/freemail/utils/EmailAddress.java	2008-05-14 21:03:03 UTC (rev 19932)
+++ trunk/apps/Freemail/src/freemail/utils/EmailAddress.java	2008-05-14 22:00:44 UTC (rev 19933)
@@ -33,8 +33,7 @@
 	public String user;
 	public String domain;
 	
-	public EmailAddress(String rawAddress) {
-		String address = rawAddress.toLowerCase();
+	public EmailAddress(String address) {
 		this.realname = null;
 		this.user = null;
 		this.domain = null;
@@ -45,7 +44,7 @@
 			
 			switch (c) {
 				case '@':
-					this.user = bank.toString();
+					this.user = bank.toString().toLowerCase();
 					bank = new StringBuffer("");
 					break;
 				case '<':
@@ -53,11 +52,11 @@
 					bank = new StringBuffer("");
 					break;
 				case '>':
-					this.domain = bank.toString();
+					this.domain = bank.toString().toLowerCase();
 					bank = new StringBuffer("");
 					break;
 				case '(':
-					this.domain = bank.toString();
+					this.domain = bank.toString().toLowerCase();
 					bank = new StringBuffer("");
 					break;
 				case ')':
@@ -70,7 +69,7 @@
 		}
 		
 		if (this.realname == null && this.domain == null) {
-			this.domain = bank.toString();
+			this.domain = bank.toString().toLowerCase();
 		}
 		
 		// trim quotes out of the real name field



From dbkr at freenetproject.org  Wed May 14 23:27:47 2008
From: dbkr at freenetproject.org (dbkr at freenetproject.org)
Date: Wed, 14 May 2008 23:27:47 +0000 (UTC)
Subject: [Freemail] r19934 - trunk/apps/Freemail/src/freemail
Message-ID: <20080514232747.380AE478FA1@freenetproject.org>

Author: dbkr
Date: 2008-05-14 23:27:47 +0000 (Wed, 14 May 2008)
New Revision: 19934

Modified:
   trunk/apps/Freemail/src/freemail/InboundContact.java
Log:
Non-freemail addresses are not valid.


Modified: trunk/apps/Freemail/src/freemail/InboundContact.java
===================================================================
--- trunk/apps/Freemail/src/freemail/InboundContact.java	2008-05-14 22:00:44 UTC (rev 19933)
+++ trunk/apps/Freemail/src/freemail/InboundContact.java	2008-05-14 23:27:47 UTC (rev 19934)
@@ -203,6 +203,11 @@
 			return false;
 		}
 		
+		if (!from.is_freemail_address()) {
+			// again, I believe this is what the Americans might call a 'no brainer'.
+			return false;
+		}
+		
 		if (from.is_ssk_address()) {
 			return Base32.encode(this.ibct_dir.getName().getBytes()).equalsIgnoreCase(sd);
 		} else {



From dbkr at freenetproject.org  Wed May 14 23:28:56 2008
From: dbkr at freenetproject.org (dbkr at freenetproject.org)
Date: Wed, 14 May 2008 23:28:56 +0000 (UTC)
Subject: [Freemail] r19935 - trunk/apps/Freemail/src/freemail
Message-ID: <20080514232856.A604D478FBC@freenetproject.org>

Author: dbkr
Date: 2008-05-14 23:28:56 +0000 (Wed, 14 May 2008)
New Revision: 19935

Modified:
   trunk/apps/Freemail/src/freemail/OutboundContact.java
Log:
Handle collisions on message inserts (should never happen under normal circumstances, but let's do it for good measure). Also add the error code for a failed message insert.


Modified: trunk/apps/Freemail/src/freemail/OutboundContact.java
===================================================================
--- trunk/apps/Freemail/src/freemail/OutboundContact.java	2008-05-14 23:27:47 UTC (rev 19934)
+++ trunk/apps/Freemail/src/freemail/OutboundContact.java	2008-05-14 23:28:56 UTC (rev 19935)
@@ -720,6 +720,10 @@
 					msgs[i].first_send_time = System.currentTimeMillis();
 				msgs[i].last_send_time = System.currentTimeMillis();
 				msgs[i].saveProps();
+			} else if (err.errorcode == FCPInsertErrorMessage.COLLISION) {
+				msgs[i].slot = popNextSlot();
+				Logger.error(this, "Insert collided! Assigned new slot: "+msgs[i].slot);
+				msgs[i].saveProps();
 			} else if (msgs[i].added_time + FAIL_DELAY < System.currentTimeMillis()) {
 				Logger.normal(this,"Giving up on a message - been trying to send for too long. Bouncing.");
 				if (Postman.bounceMessage(msgs[i].getMessageFile(), account.getMessageBank(),
@@ -728,7 +732,7 @@
 					msgs[i].delete();
 				}
 			} else {
-				Logger.normal(this,"Failed to insert "+key+" will try again soon.");
+				Logger.normal(this,"Failed to insert "+key+" (error code "+err.errorcode+") will try again soon.");
 			}
 		}
 	}



From alexlehm at myrealbox.com  Sun May 18 09:54:16 2008
From: alexlehm at myrealbox.com (Alexander Lehmann)
Date: Sun, 18 May 2008 11:54:16 +0200
Subject: [Freemail] PropFiles conflicting writes problem
Message-ID: <1211104456.400f65dcalexlehm@myrealbox.com>

what is your opinion, should I commit this and leave the memory issue
for later or should I first write a fix for this?

https://bugs.freenetproject.org/view.php?id=2377

If at all this will only be a problem for people that receive and send
lots of mails if they never restart Freemail or the node.

bye, Alexander

-- 
Alexander Lehmann <alexlehm at myrealbox.com>




From dbkr at freenetproject.org  Mon May 19 08:23:38 2008
From: dbkr at freenetproject.org (Dave Baker)
Date: Mon, 19 May 2008 09:23:38 +0100
Subject: [Freemail] PropFiles conflicting writes problem
In-Reply-To: <1211104456.400f65dcalexlehm@myrealbox.com>
References: <1211104456.400f65dcalexlehm@myrealbox.com>
Message-ID: <200805190923.38811.dbkr@freenetproject.org>

On Sunday 18 May 2008 10:54:16 Alexander Lehmann wrote:
> what is your opinion, should I commit this and leave the memory issue
> for later or should I first write a fix for this?
>
> https://bugs.freenetproject.org/view.php?id=2377
>
> If at all this will only be a problem for people that receive and send
> lots of mails if they never restart Freemail or the node.
>
> bye, Alexander

Hi,

I tried to get around this issue with the account properties files by opening 
the objects once and then passing them around. I was contemplating a solution 
like this as it does cover the more general problem and make sure we don't 
get conflicting writes for any props files, but of course it comes with its 
own complications as you rightly point out.

I'm becoming more convinced that this is a better solution now, so if you 
commit your changes, I'm sure we can fix the memory leak relatively simply. 
Provided we always delete props files through the PropsFile object, we can 
remove the entries when the file is deleted.


Dave


From alexlehm at freenetproject.org  Tue May 20 20:24:40 2008
From: alexlehm at freenetproject.org (alexlehm at freenetproject.org)
Date: Tue, 20 May 2008 20:24:40 +0000 (UTC)
Subject: [Freemail] r19976 - in trunk/apps/Freemail/src/freemail: . config
	utils
Message-ID: <20080520202440.D3F79478EF7@freenetproject.org>

Author: alexlehm
Date: 2008-05-20 20:24:40 +0000 (Tue, 20 May 2008)
New Revision: 19976

Modified:
   trunk/apps/Freemail/src/freemail/AccountManager.java
   trunk/apps/Freemail/src/freemail/AckProcrastinator.java
   trunk/apps/Freemail/src/freemail/InboundContact.java
   trunk/apps/Freemail/src/freemail/OutboundContact.java
   trunk/apps/Freemail/src/freemail/RTSFetcher.java
   trunk/apps/Freemail/src/freemail/RTSLog.java
   trunk/apps/Freemail/src/freemail/config/Configurator.java
   trunk/apps/Freemail/src/freemail/utils/PropsFile.java
Log:
0002377: Parent issue for PropsFile conflicting writes problems. (https://bugs.freenetproject.org/view.php?id=2377)
resp the child issues
0002344 sending many mails immediately after each makes freemail loose some slots
0002176 Messages are sometimes inserted more than once
0001989 Slot with SSK at key/null

changed PropsFile to a factory with Eclipse, we only instantiate every PropsFile once by filename, this should fix the conflicting write problems, but creates a minor memory leak with temp files, as mentioned before. We will fix this later.

(Also, I fixed some typos in comments and message strings.)


Modified: trunk/apps/Freemail/src/freemail/AccountManager.java
===================================================================
--- trunk/apps/Freemail/src/freemail/AccountManager.java	2008-05-20 16:30:25 UTC (rev 19975)
+++ trunk/apps/Freemail/src/freemail/AccountManager.java	2008-05-20 20:24:40 UTC (rev 19976)
@@ -183,7 +183,7 @@
 	}
 	
 	private static PropsFile getAccountFile(File accdir) {
-		PropsFile accfile = new PropsFile(new File(accdir, ACCOUNT_FILE));
+		PropsFile accfile = PropsFile.createPropsFile(new File(accdir, ACCOUNT_FILE));
 		
 		if (!accdir.exists() || !accfile.exists()) {
 			return null;
@@ -193,7 +193,7 @@
 	}
 	
 	private static PropsFile newAccountFile(File accdir) {
-		PropsFile accfile = new PropsFile(new File(accdir, ACCOUNT_FILE));
+		PropsFile accfile = PropsFile.createPropsFile(new File(accdir, ACCOUNT_FILE));
 		
 		if (accdir.exists() && !accfile.exists()) {
 			initAccFile(accfile);

Modified: trunk/apps/Freemail/src/freemail/AckProcrastinator.java
===================================================================
--- trunk/apps/Freemail/src/freemail/AckProcrastinator.java	2008-05-20 16:30:25 UTC (rev 19975)
+++ trunk/apps/Freemail/src/freemail/AckProcrastinator.java	2008-05-20 20:24:40 UTC (rev 19976)
@@ -37,8 +37,8 @@
 
 /** Takes simple pieces of data to insert to keys and inserts them at some point
  * randomly within a given time frame in order to disguise the time at which messages
- * were received. This is by no means infalliable, and will only work effectively if
- * Freemail is run more or less permanantly.
+ * were received. This is by no means infallible, and will only work effectively if
+ * Freemail is run more or less permanently.
  */
 public class AckProcrastinator implements Runnable {
 	/**
@@ -78,7 +78,7 @@
 			
 			int i;
 			for (i = 0; i < acks.length; i++) {
-				PropsFile ack = new PropsFile(acks[i]);
+				PropsFile ack = PropsFile.createPropsFile(acks[i]);
 				
 				String s_it  = ack.get("nominalInsertTime");
 				String key = ack.get("key");
@@ -108,10 +108,10 @@
 						FCPInsertErrorMessage err = fcpcli.put(bis, key);
 						if (err == null) {
 							acks[i].delete();
-							Logger.normal(this,"ACK insertion to "+key+" sucessful");
+							Logger.normal(this,"ACK insertion to "+key+" successful");
 						} else if (err.errorcode == FCPInsertErrorMessage.COLLISION) {
 							acks[i].delete();
-							Logger.normal(this,"ACK insertion to "+key+" sucessful");
+							Logger.normal(this,"ACK insertion to "+key+" successful");
 						}
 					} catch (FCPBadFileException bfe) {
 						// won't occur
@@ -154,7 +154,7 @@
 		long by = System.currentTimeMillis() + MAX_DELAY;
 		
 		try {
-			PropsFile ackfile= new PropsFile(File.createTempFile("delayed-ack", "", getAckDir()));
+			PropsFile ackfile= PropsFile.createPropsFile(File.createTempFile("delayed-ack", "", getAckDir()));
 			 
 			 ackfile.put("key", key);
 			 if (data != null)

Modified: trunk/apps/Freemail/src/freemail/InboundContact.java
===================================================================
--- trunk/apps/Freemail/src/freemail/InboundContact.java	2008-05-20 16:30:25 UTC (rev 19975)
+++ trunk/apps/Freemail/src/freemail/InboundContact.java	2008-05-20 20:24:40 UTC (rev 19976)
@@ -58,7 +58,7 @@
 			this.ibct_dir.mkdir();
 		}
 		
-		this.ibct_props = new PropsFile(new File(this.ibct_dir, IBCT_PROPSFILE));
+		this.ibct_props = PropsFile.createPropsFile(new File(this.ibct_dir, IBCT_PROPSFILE));
 	}
 	
 	public void setProp(String key, String val) {
@@ -115,7 +115,7 @@
 			Logger.normal(this,"Found a message!");
 			
 			// parse the Freemail header(s) out.
-			PropsFile msgprops = new PropsFile(msg, true);
+			PropsFile msgprops = PropsFile.createPropsFile(msg, true);
 			String s_id = msgprops.get("id");
 			if (s_id == null) {
 				Logger.error(this,"Got a message with an invalid header. Discarding.");
@@ -199,7 +199,7 @@
 	public boolean validateFrom(EmailAddress from) throws IOException, ConnectionTerminatedException {
 		String sd = from.getSubDomain();
 		if (sd == null) {
-			// well that's definately not valid. Piffle!
+			// well that's definitely not valid. Piffle!
 			return false;
 		}
 		

Modified: trunk/apps/Freemail/src/freemail/OutboundContact.java
===================================================================
--- trunk/apps/Freemail/src/freemail/OutboundContact.java	2008-05-20 16:30:25 UTC (rev 19975)
+++ trunk/apps/Freemail/src/freemail/OutboundContact.java	2008-05-20 20:24:40 UTC (rev 19976)
@@ -139,7 +139,7 @@
 				throw new IOException("Couldn't create outbound contact dir!");
 			}
 			
-			this.contactfile = new PropsFile(new File(obctdir, PROPSFILE_NAME));
+			this.contactfile = PropsFile.createPropsFile(new File(obctdir, PROPSFILE_NAME));
 			this.ctoutbox = new File(obctdir, OUTBOX_DIR);
 			if (!this.ctoutbox.exists() && !this.ctoutbox.mkdir()) {
 				throw new IOException("Couldn't create contact outbox!");
@@ -152,7 +152,7 @@
 		this.address = new EmailAddress();
 		this.address.domain = ctdir.getName()+".freemail";
 		
-		this.contactfile = new PropsFile(new File(ctdir, PROPSFILE_NAME));
+		this.contactfile = PropsFile.createPropsFile(new File(ctdir, PROPSFILE_NAME));
 		
 		this.ctoutbox = new File(ctdir, OUTBOX_DIR);
 		if (!this.ctoutbox.exists()) {
@@ -540,7 +540,7 @@
 		
 		Logger.normal(this,"got mailsite");
 		
-		PropsFile mailsite = new PropsFile(mailsite_file);
+		PropsFile mailsite = PropsFile.createPropsFile(mailsite_file);
 		
 		String rtsksk = mailsite.get("rtsksk");
 		String keymod_str = mailsite.get("asymkey.modulus");
@@ -834,7 +834,7 @@
 			this.uid = uid;
 			this.file = new File(ctoutbox, Integer.toString(uid));
 			
-			this.index = new PropsFile(new File(ctoutbox, INDEX_FILE));
+			this.index = PropsFile.createPropsFile(file);
 			
 			this.slot = this.index.get(uid+".slot");
 			String s_first = this.index.get(uid+".first_send_time");

Modified: trunk/apps/Freemail/src/freemail/RTSFetcher.java
===================================================================
--- trunk/apps/Freemail/src/freemail/RTSFetcher.java	2008-05-20 16:30:25 UTC (rev 19975)
+++ trunk/apps/Freemail/src/freemail/RTSFetcher.java	2008-05-20 20:24:40 UTC (rev 19976)
@@ -288,7 +288,7 @@
 			return false;
 		}
 		
-		PropsFile rtsprops = new PropsFile(rtsfile);
+		PropsFile rtsprops = PropsFile.createPropsFile(rtsfile);
 		
 		try {
 			validate_rts(rtsprops);
@@ -335,7 +335,7 @@
 			return false;
 		}
 		
-		PropsFile mailsite = new PropsFile(msfile);
+		PropsFile mailsite = PropsFile.createPropsFile(msfile);
 		String their_exponent = mailsite.get("asymkey.pubexponent");
 		String their_modulus = mailsite.get("asymkey.modulus");
 		

Modified: trunk/apps/Freemail/src/freemail/RTSLog.java
===================================================================
--- trunk/apps/Freemail/src/freemail/RTSLog.java	2008-05-20 16:30:25 UTC (rev 19975)
+++ trunk/apps/Freemail/src/freemail/RTSLog.java	2008-05-20 20:24:40 UTC (rev 19976)
@@ -39,7 +39,7 @@
 	private static String UNPROC_NEXTID = "unproc-nextid";
 
 	public RTSLog(File f) {
-		this.logfile = new PropsFile(f);
+		this.logfile = PropsFile.createPropsFile(f);
 		if (!this.logfile.exists()) {
 			String birth = DateStringFactory.getOffsetKeyString(0);
 			this.logfile.put("birth", birth);

Modified: trunk/apps/Freemail/src/freemail/config/Configurator.java
===================================================================
--- trunk/apps/Freemail/src/freemail/config/Configurator.java	2008-05-20 16:30:25 UTC (rev 19975)
+++ trunk/apps/Freemail/src/freemail/config/Configurator.java	2008-05-20 20:24:40 UTC (rev 19976)
@@ -35,7 +35,7 @@
 	private final HashMap callbacks;
 
 	public Configurator(File f) {
-		this.props = new PropsFile(f);
+		this.props = PropsFile.createPropsFile(f);
 		this.props.setCommentPrefix("#");
 		String ls = System.getProperty("line.separator");
 		StringBuffer head = new StringBuffer();

Modified: trunk/apps/Freemail/src/freemail/utils/PropsFile.java
===================================================================
--- trunk/apps/Freemail/src/freemail/utils/PropsFile.java	2008-05-20 16:30:25 UTC (rev 19975)
+++ trunk/apps/Freemail/src/freemail/utils/PropsFile.java	2008-05-20 20:24:40 UTC (rev 19976)
@@ -31,8 +31,31 @@
 import java.util.Iterator;
 import java.util.Map;
 import java.util.Set;
+import java.util.Hashtable;
 
 public class PropsFile {
+	// substitute static methods for constructor
+	
+	static Hashtable propsList=new Hashtable();
+	
+	public static PropsFile createPropsFile(File f, boolean stopAtBlank) {
+		String fn=f.getPath();
+
+		PropsFile pf=(PropsFile)propsList.get(fn);
+		
+		if(pf!=null) {
+			return pf;
+		} else {
+			pf=new PropsFile(f, stopAtBlank);
+			propsList.put(fn, pf);
+			return pf;
+		}
+	}
+
+	public static PropsFile createPropsFile(File f) {
+		return createPropsFile(f, false);
+	}
+
 	private final File file;
 	private HashMap data;
 	private BufferedReader bufrdr;
@@ -43,7 +66,7 @@
 	 * a blank line. It's the the caller's responsibility to get
 	 * (using the getReader() method) the stream and close it properly.
 	 */
-	public PropsFile(File f, boolean stopAtBlank) {
+	private PropsFile(File f, boolean stopAtBlank) {
 		this.file = f;
 		this.data = null;
 		
@@ -57,10 +80,6 @@
 		this.header = null;
 	}
 	
-	public PropsFile(File f) {
-		this(f, false);
-	}
-	
 	public void setCommentPrefix(String cp) {
 		this.commentPrefix = cp;
 	}



From alexlehm at freenetproject.org  Tue May 20 20:58:03 2008
From: alexlehm at freenetproject.org (alexlehm at freenetproject.org)
Date: Tue, 20 May 2008 20:58:03 +0000 (UTC)
Subject: [Freemail] r19977 - trunk/apps/Freemail/src/freemail/imap
Message-ID: <20080520205803.BB48047904D@freenetproject.org>

Author: alexlehm
Date: 2008-05-20 20:58:03 +0000 (Tue, 20 May 2008)
New Revision: 19977

Modified:
   trunk/apps/Freemail/src/freemail/imap/IMAPMessageFlags.java
Log:
0002264: when deleting more than one message in Thunderbird at once, they are not marked read (https://bugs.freenetproject.org/view.php?id=2264)

just mark messages that are set \\Deleted as \\Seen as well

Modified: trunk/apps/Freemail/src/freemail/imap/IMAPMessageFlags.java
===================================================================
--- trunk/apps/Freemail/src/freemail/imap/IMAPMessageFlags.java	2008-05-20 20:24:40 UTC (rev 19976)
+++ trunk/apps/Freemail/src/freemail/imap/IMAPMessageFlags.java	2008-05-20 20:58:03 UTC (rev 19977)
@@ -104,6 +104,9 @@
 		
 		if (value) {
 			this.flags.add(flag);
+			if(flag.equals("\\Deleted")) {
+				set("\\Seen", true);
+			}
 		} else {
 			this.flags.remove(flag);
 		}



From alexlehm at freenetproject.org  Tue May 20 21:06:30 2008
From: alexlehm at freenetproject.org (alexlehm at freenetproject.org)
Date: Tue, 20 May 2008 21:06:30 +0000 (UTC)
Subject: [Freemail] r19978 - trunk/apps/Freemail/src/freemail
Message-ID: <20080520210630.3E0CD479074@freenetproject.org>

Author: alexlehm
Date: 2008-05-20 21:06:30 +0000 (Tue, 20 May 2008)
New Revision: 19978

Modified:
   trunk/apps/Freemail/src/freemail/SingleAccountWatcher.java
Log:
log the different stages (logged as DEBUG, unless you are developing Freemail you will never see this)

Modified: trunk/apps/Freemail/src/freemail/SingleAccountWatcher.java
===================================================================
--- trunk/apps/Freemail/src/freemail/SingleAccountWatcher.java	2008-05-20 20:58:03 UTC (rev 19977)
+++ trunk/apps/Freemail/src/freemail/SingleAccountWatcher.java	2008-05-20 21:06:30 UTC (rev 19978)
@@ -117,6 +117,7 @@
 					break;
 				}
 				// send any messages queued in contact outboxes
+				Logger.debug(this, "sending any message in contact outboxes");
 				File[] obcontacts = this.obctdir.listFiles(new outboundContactFilenameFilter());
 				if (obcontacts != null) {
 					int i;
@@ -129,6 +130,7 @@
 						}
 					}
 				}
+				Logger.debug(this, "polling rts");
 				if (this.nf != null) {
 					nf.fetch();
 				}
@@ -138,6 +140,7 @@
 				}
 				
 				// poll for incoming message from all inbound contacts
+				Logger.debug(this, "polling for incoming message from all inbound contacts");
 				File[] ibcontacts = this.ibctdir.listFiles(new inboundContactFilenameFilter());
 				if (ibcontacts != null) {
 					int i;



From alexlehm at freenetproject.org  Tue May 20 21:10:04 2008
From: alexlehm at freenetproject.org (alexlehm at freenetproject.org)
Date: Tue, 20 May 2008 21:10:04 +0000 (UTC)
Subject: [Freemail] r19979 - trunk/apps/Freemail/src/freemail
Message-ID: <20080520211004.9DA8B478D54@freenetproject.org>

Author: alexlehm
Date: 2008-05-20 21:10:04 +0000 (Tue, 20 May 2008)
New Revision: 19979

Modified:
   trunk/apps/Freemail/src/freemail/AckProcrastinator.java
   trunk/apps/Freemail/src/freemail/MailSite.java
   trunk/apps/Freemail/src/freemail/OutboundContact.java
Log:
make a few error message more descriptive, mention the error code if we do not mention e.g. collision
catch a null string access if the slot entry in undefined


Modified: trunk/apps/Freemail/src/freemail/AckProcrastinator.java
===================================================================
--- trunk/apps/Freemail/src/freemail/AckProcrastinator.java	2008-05-20 21:06:30 UTC (rev 19978)
+++ trunk/apps/Freemail/src/freemail/AckProcrastinator.java	2008-05-20 21:10:04 UTC (rev 19979)
@@ -112,6 +112,8 @@
 						} else if (err.errorcode == FCPInsertErrorMessage.COLLISION) {
 							acks[i].delete();
 							Logger.normal(this,"ACK insertion to "+key+" successful");
+						} else {
+							Logger.error(this,"ACK insertion to "+key+" failed (Errorcode: "+err.errorcode+")");
 						}
 					} catch (FCPBadFileException bfe) {
 						// won't occur

Modified: trunk/apps/Freemail/src/freemail/MailSite.java
===================================================================
--- trunk/apps/Freemail/src/freemail/MailSite.java	2008-05-20 21:06:30 UTC (rev 19978)
+++ trunk/apps/Freemail/src/freemail/MailSite.java	2008-05-20 21:10:04 UTC (rev 19979)
@@ -147,8 +147,11 @@
 		} else if (err.errorcode == FCPInsertErrorMessage.COLLISION) {
 			Logger.error(this,"Mailsite alias collided - somebody is already using that alias! Choose another one!");
 			return false;
+		} else if (err.errorcode == FCPInsertErrorMessage.REJECTED_OVERLOAD) {
+			Logger.error(this,"Mailsite alias could not be inserted (rejected overload), this is probably a temporary error");
+			return false;
 		} else {
-			Logger.error(this,"Mailsite redirect insert failed, but did not collide.");
+			Logger.error(this,"Mailsite redirect insert failed, but did not collide. (errorcode="+err.errorcode+")");
 			return false;
 		}
 	}

Modified: trunk/apps/Freemail/src/freemail/OutboundContact.java
===================================================================
--- trunk/apps/Freemail/src/freemail/OutboundContact.java	2008-05-20 21:06:30 UTC (rev 19978)
+++ trunk/apps/Freemail/src/freemail/OutboundContact.java	2008-05-20 21:10:04 UTC (rev 19979)
@@ -697,6 +697,12 @@
 				continue;
 			}
 			
+			if(msgs[i].slot==null) {
+				Logger.normal(this,"Index file does not contain slot name for this message, the mail cannot be sent this way.");
+				Logger.debug(this,"Filename is "+contactfile.getFile().getPath());
+				continue;
+			}
+			
 			key += msgs[i].slot;
 			
 			FileInputStream fis;
@@ -733,6 +739,11 @@
 				}
 			} else {
 				Logger.normal(this,"Failed to insert "+key+" (error code "+err.errorcode+") will try again soon.");
+				if(err.errorcode==FCPInsertErrorMessage.COLLISION) {
+					Logger.error(this,"Failed to insert "+key+" will try again soon. (Collision, this shouldn't happen)");
+				} else {
+					Logger.normal(this,"Failed to insert "+key+" will try again soon. Error: "+err.errorcode);
+				}
 			}
 		}
 	}



From dbkr at freenetproject.org  Tue May 20 21:22:04 2008
From: dbkr at freenetproject.org (dbkr at freenetproject.org)
Date: Tue, 20 May 2008 21:22:04 +0000 (UTC)
Subject: [Freemail] r19980 - in trunk/apps/Freemail/src/freemail: . utils
Message-ID: <20080520212204.C9B92478E02@freenetproject.org>

Author: dbkr
Date: 2008-05-20 21:22:04 +0000 (Tue, 20 May 2008)
New Revision: 19980

Modified:
   trunk/apps/Freemail/src/freemail/OutboundContact.java
   trunk/apps/Freemail/src/freemail/utils/PropsFile.java
Log:
Fix the build (seems reasonable that PropsFile should have a toString() for printing logging strings).


Modified: trunk/apps/Freemail/src/freemail/OutboundContact.java
===================================================================
--- trunk/apps/Freemail/src/freemail/OutboundContact.java	2008-05-20 21:10:04 UTC (rev 19979)
+++ trunk/apps/Freemail/src/freemail/OutboundContact.java	2008-05-20 21:22:04 UTC (rev 19980)
@@ -699,7 +699,7 @@
 			
 			if(msgs[i].slot==null) {
 				Logger.normal(this,"Index file does not contain slot name for this message, the mail cannot be sent this way.");
-				Logger.debug(this,"Filename is "+contactfile.getFile().getPath());
+				Logger.debug(this,"Filename is "+contactfile.toString());
 				continue;
 			}
 			

Modified: trunk/apps/Freemail/src/freemail/utils/PropsFile.java
===================================================================
--- trunk/apps/Freemail/src/freemail/utils/PropsFile.java	2008-05-20 21:10:04 UTC (rev 19979)
+++ trunk/apps/Freemail/src/freemail/utils/PropsFile.java	2008-05-20 21:22:04 UTC (rev 19980)
@@ -182,4 +182,8 @@
 		}
 		return true;
 	}
+	
+	public String toString() {
+		return file.getPath();
+	}
 }



From dbkr at freenetproject.org  Tue May 20 21:53:09 2008
From: dbkr at freenetproject.org (dbkr at freenetproject.org)
Date: Tue, 20 May 2008 21:53:09 +0000 (UTC)
Subject: [Freemail] r19982 - trunk/apps/Freemail/src/freemail/utils
Message-ID: <20080520215309.EC3ED478F37@freenetproject.org>

Author: dbkr
Date: 2008-05-20 21:53:09 +0000 (Tue, 20 May 2008)
New Revision: 19982

Modified:
   trunk/apps/Freemail/src/freemail/utils/PropsFile.java
Log:
Really simplistic reaping of stale (deleted) PropsFiles to address the memory leak issue.


Modified: trunk/apps/Freemail/src/freemail/utils/PropsFile.java
===================================================================
--- trunk/apps/Freemail/src/freemail/utils/PropsFile.java	2008-05-20 21:46:28 UTC (rev 19981)
+++ trunk/apps/Freemail/src/freemail/utils/PropsFile.java	2008-05-20 21:53:09 UTC (rev 19982)
@@ -36,9 +36,20 @@
 public class PropsFile {
 	// substitute static methods for constructor
 	
-	static Hashtable propsList=new Hashtable();
+	private static final Hashtable propsList=new Hashtable();
 	
-	public static PropsFile createPropsFile(File f, boolean stopAtBlank) {
+	private static int reapCounter = 0;
+	/// We go through the list and remove stale entries once in this many times a PropsFile is created
+	private static final int reapEvery = 20;
+	
+	public static synchronized PropsFile createPropsFile(File f, boolean stopAtBlank) {
+		if (reapCounter == reapEvery) {
+			reapOld();
+			reapCounter = 0;
+		} else {
+			++reapCounter;
+		}
+		
 		String fn=f.getPath();
 
 		PropsFile pf=(PropsFile)propsList.get(fn);
@@ -55,6 +66,21 @@
 	public static PropsFile createPropsFile(File f) {
 		return createPropsFile(f, false);
 	}
+	
+	public static void reapOld() {
+		Logger.debug(PropsFile.class, "Cleaning up stale PropsFiles");
+		
+		Iterator i = propsList.entrySet().iterator();
+		
+		while (i.hasNext()) {
+			Map.Entry entry = (Map.Entry)i.next();
+			File f = new File((String)entry.getKey());
+			if (!f.exists()) {
+				Logger.debug(PropsFile.class, "Removing "+f.getPath());
+				i.remove();
+			}
+		}
+	}
 
 	private final File file;
 	private HashMap data;



From alexlehm at freenetproject.org  Tue May 20 22:07:32 2008
From: alexlehm at freenetproject.org (alexlehm at freenetproject.org)
Date: Tue, 20 May 2008 22:07:32 +0000 (UTC)
Subject: [Freemail] r19984 - trunk/apps/Freemail/src/freemail
Message-ID: <20080520220732.93D84478EEC@freenetproject.org>

Author: alexlehm
Date: 2008-05-20 22:07:32 +0000 (Tue, 20 May 2008)
New Revision: 19984

Modified:
   trunk/apps/Freemail/src/freemail/OutboundContact.java
Log:
fix error in commit r19976, the _index PropsFile is initialized with message file instead


Modified: trunk/apps/Freemail/src/freemail/OutboundContact.java
===================================================================
--- trunk/apps/Freemail/src/freemail/OutboundContact.java	2008-05-20 22:06:18 UTC (rev 19983)
+++ trunk/apps/Freemail/src/freemail/OutboundContact.java	2008-05-20 22:07:32 UTC (rev 19984)
@@ -845,7 +845,7 @@
 			this.uid = uid;
 			this.file = new File(ctoutbox, Integer.toString(uid));
 			
-			this.index = PropsFile.createPropsFile(file);
+			this.index = PropsFile.createPropsFile(new File(ctoutbox, INDEX_FILE));
 			
 			this.slot = this.index.get(uid+".slot");
 			String s_first = this.index.get(uid+".first_send_time");



From alexlehm at freenetproject.org  Thu May 22 12:15:45 2008
From: alexlehm at freenetproject.org (alexlehm at freenetproject.org)
Date: Thu, 22 May 2008 12:15:45 +0000 (UTC)
Subject: [Freemail] r20029 - trunk/apps/Freemail/src/freemail
Message-ID: <20080522121545.65744478E5F@freenetproject.org>

Author: alexlehm
Date: 2008-05-22 12:15:45 +0000 (Thu, 22 May 2008)
New Revision: 20029

Modified:
   trunk/apps/Freemail/src/freemail/Postman.java
Log:
0002384: Postmaster mails use a localized date string (https://bugs.freenetproject.org/view.php?id=2384)


Modified: trunk/apps/Freemail/src/freemail/Postman.java
===================================================================
--- trunk/apps/Freemail/src/freemail/Postman.java	2008-05-22 10:44:51 UTC (rev 20028)
+++ trunk/apps/Freemail/src/freemail/Postman.java	2008-05-22 12:15:45 UTC (rev 20029)
@@ -23,6 +23,7 @@
 
 import java.text.SimpleDateFormat;
 import java.util.Date;
+import java.util.Locale;
 import java.util.Random;
 import java.io.File;
 import java.io.BufferedReader;
@@ -46,7 +47,7 @@
 	protected void storeMessage(BufferedReader brdr, MessageBank mb) throws IOException, ConnectionTerminatedException {
 		MailMessage newmsg = mb.createMessage();
 		
-		SimpleDateFormat sdf = new SimpleDateFormat("dd MMM yyyy HH:mm:ss Z");
+		SimpleDateFormat sdf = new SimpleDateFormat("dd MMM yyyy HH:mm:ss Z", Locale.US);
 		
 		newmsg.readHeaders(brdr);
 		
@@ -98,7 +99,7 @@
 		try {
 			bmsg = mb.createMessage();
 			
-			SimpleDateFormat sdf = new SimpleDateFormat("dd MMM yyyy HH:mm:ss Z");
+			SimpleDateFormat sdf = new SimpleDateFormat("dd MMM yyyy HH:mm:ss Z", Locale.US);
 			
 			bmsg.addHeader("From", "Freemail Postmaster <postmaster at freemail>");
 			bmsg.addHeader("Subject", "Undeliverable Freemail");



From dave at accidentalegg.co.uk  Sun May  4 20:06:32 2008
From: dave at accidentalegg.co.uk (Dave Baker)
Date: Sun, 04 May 2008 20:06:32 -0000
Subject: [Freemail] Log: Don't re-insert mail to different slots
In-Reply-To: <1209823290.c8047d7calexlehm@myrealbox.com>
References: <1209823290.c8047d7calexlehm@myrealbox.com>
Message-ID: <200805042105.49681.dave@accidentalegg.co.uk>

On Saturday 03 May 2008 15:01:30 Alexander Lehmann wrote:
> I think the rationale behind polling ahead on the hash slots was that the 
user may miss an inserted key sometimes, if this is the case, it would be 
useful to insert the message to a new slot when starting a new rts, it that 
cannot happen, it is not necessary since the old key can be used.
> 
> However if we do not expect this to happen, do we really need to poll ahead 
on the keys?

Now you come to mention it, it ought not to be necessary - pehaps we can 
remove it, or at least decrease it. As far as I can see, the only reasons 
we'd find messages on a slot having not found any on a previous slot would be 
if there were a problem with sngle-recipient keys on Freenet (in which case 
we're screwed anyway) or if there's a bug.

I think some backwards incompatable changes to RTS messages will be in order 
in the near future, so I think we'll be gaining a 'development version'. 
Perhaps we could disable poll-ahead in the dev version?


Dave


> 
> > Message: 2
> > Date: Thu, 1 May 2008 16:23:59 +0100
> > From: Dave Baker <dbkr at freenetproject.org>
> > Subject: Re: [Freemail] Log: Don't re-insert mail to different slots
> > To: freemail at freenetproject.org
> > Message-ID: <200805011623.59719.dbkr at freenetproject.org>
> > Content-Type: text/plain;  charset="iso-8859-1"
> > 
> > On Thursday 01 May 2008 16:01:47 Alexander Lehmann wrote:
> > > I think it may be useful to first try to insert the message to a new 
slot
> > > (in case the user has missed a slot somehow) and later to retry rts or 
is
> > > this a situation that doesn't seem likely.
> > >
> > > freemail-request at freenetproject.org wrote:
> > > > Modified:
> > > >    trunk/apps/Freemail/src/freemail/OutboundContact.java
> > > > Log:
> > > > Don't re-insert mail to different slots - re-send the RTS instead. 
This
> > > > will allow contacts to be re-established if the inbound contact has 
been
> > > > deleted.
> > 
> > Well, I was thinking along the lines that the user missing a slot 
shouldn't 
> > happen, since we'll only advance through a slot if we find a message on 
it. I 
> > don't think it's every likley to be that a user cann't fetch one ket but 
can 
> > fetch the same data on a different key.
> > 
> > Feedback welcome though.
> > 
> > 
> > Dave
> 
> 
> -- 
> Alexander Lehmann <alexlehm at myrealbox.com>
> 
> 
> _______________________________________________
> Freemail mailing list
> Freemail at freenetproject.org
> http://emu.freenetproject.org/cgi-bin/mailman/listinfo/freemail
> 




From dave at accidentalegg.co.uk  Tue May 20 21:57:25 2008
From: dave at accidentalegg.co.uk (Dave Baker)
Date: Tue, 20 May 2008 21:57:25 -0000
Subject: [Freemail] r19976 - in trunk/apps/Freemail/src/freemail: .
	config utils
In-Reply-To: <20080520202440.D3F79478EF7@freenetproject.org>
References: <20080520202440.D3F79478EF7@freenetproject.org>
Message-ID: <200805202257.17989.dave@accidentalegg.co.uk>

On Tuesday 20 May 2008 21:24:40 alexlehm at freenetproject.org wrote:
> Author: alexlehm
> Date: 2008-05-20 20:24:40 +0000 (Tue, 20 May 2008)
> New Revision: 19976
> 
> Modified:
>    trunk/apps/Freemail/src/freemail/AccountManager.java
>    trunk/apps/Freemail/src/freemail/AckProcrastinator.java
>    trunk/apps/Freemail/src/freemail/InboundContact.java
>    trunk/apps/Freemail/src/freemail/OutboundContact.java
>    trunk/apps/Freemail/src/freemail/RTSFetcher.java
>    trunk/apps/Freemail/src/freemail/RTSLog.java
>    trunk/apps/Freemail/src/freemail/config/Configurator.java
>    trunk/apps/Freemail/src/freemail/utils/PropsFile.java
> Log:
> 0002377: Parent issue for PropsFile conflicting writes problems. 
(https://bugs.freenetproject.org/view.php?id=2377)
> resp the child issues
> 0002344 sending many mails immediately after each makes freemail loose some 
slots
> 0002176 Messages are sometimes inserted more than once
> 0001989 Slot with SSK at key/null
> 
> changed PropsFile to a factory with Eclipse, we only instantiate every 
PropsFile once by filename, this should fix the conflicting write problems, 
but creates a minor memory leak with temp files, as mentioned before. We will 
fix this later.
> 
> (Also, I fixed some typos in comments and message strings.)

Great stuff - thanks. I've just added a reaper method that gets called every 
20 PropsFile creations which will address the memory leak issue.

Looks good though, thanks for that.


Dave

PS. You broke the build in r19979 - nothing major but try to be on IRC when 
you commit things so you see FreenetLogBot's autobuild messge & we can shout 
at you when it goes wrong. ;)


> 
> 
> Modified: trunk/apps/Freemail/src/freemail/AccountManager.java
> ===================================================================
> --- trunk/apps/Freemail/src/freemail/AccountManager.java	2008-05-20 16:30:25 
UTC (rev 19975)
> +++ trunk/apps/Freemail/src/freemail/AccountManager.java	2008-05-20 20:24:40 
UTC (rev 19976)
> @@ -183,7 +183,7 @@
>  	}
>  	
>  	private static PropsFile getAccountFile(File accdir) {
> -		PropsFile accfile = new PropsFile(new File(accdir, ACCOUNT_FILE));
> +		PropsFile accfile = PropsFile.createPropsFile(new File(accdir, 
ACCOUNT_FILE));
>  		
>  		if (!accdir.exists() || !accfile.exists()) {
>  			return null;
> @@ -193,7 +193,7 @@
>  	}
>  	
>  	private static PropsFile newAccountFile(File accdir) {
> -		PropsFile accfile = new PropsFile(new File(accdir, ACCOUNT_FILE));
> +		PropsFile accfile = PropsFile.createPropsFile(new File(accdir, 
ACCOUNT_FILE));
>  		
>  		if (accdir.exists() && !accfile.exists()) {
>  			initAccFile(accfile);
> 
> Modified: trunk/apps/Freemail/src/freemail/AckProcrastinator.java
> ===================================================================
> --- trunk/apps/Freemail/src/freemail/AckProcrastinator.java	2008-05-20 
16:30:25 UTC (rev 19975)
> +++ trunk/apps/Freemail/src/freemail/AckProcrastinator.java	2008-05-20 
20:24:40 UTC (rev 19976)
> @@ -37,8 +37,8 @@
>  
>  /** Takes simple pieces of data to insert to keys and inserts them at some 
point
>   * randomly within a given time frame in order to disguise the time at 
which messages
> - * were received. This is by no means infalliable, and will only work 
effectively if
> - * Freemail is run more or less permanantly.
> + * were received. This is by no means infallible, and will only work 
effectively if
> + * Freemail is run more or less permanently.
>   */
>  public class AckProcrastinator implements Runnable {
>  	/**
> @@ -78,7 +78,7 @@
>  			
>  			int i;
>  			for (i = 0; i < acks.length; i++) {
> -				PropsFile ack = new PropsFile(acks[i]);
> +				PropsFile ack = PropsFile.createPropsFile(acks[i]);
>  				
>  				String s_it  = ack.get("nominalInsertTime");
>  				String key = ack.get("key");
> @@ -108,10 +108,10 @@
>  						FCPInsertErrorMessage err = fcpcli.put(bis, key);
>  						if (err == null) {
>  							acks[i].delete();
> -							Logger.normal(this,"ACK insertion to "+key+" sucessful");
> +							Logger.normal(this,"ACK insertion to "+key+" successful");
>  						} else if (err.errorcode == FCPInsertErrorMessage.COLLISION) {
>  							acks[i].delete();
> -							Logger.normal(this,"ACK insertion to "+key+" sucessful");
> +							Logger.normal(this,"ACK insertion to "+key+" successful");
>  						}
>  					} catch (FCPBadFileException bfe) {
>  						// won't occur
> @@ -154,7 +154,7 @@
>  		long by = System.currentTimeMillis() + MAX_DELAY;
>  		
>  		try {
> -			PropsFile ackfile= new PropsFile(File.createTempFile("delayed-ack", "", 
getAckDir()));
> +			PropsFile ackfile= 
PropsFile.createPropsFile(File.createTempFile("delayed-ack", "", 
getAckDir()));
>  			 
>  			 ackfile.put("key", key);
>  			 if (data != null)
> 
> Modified: trunk/apps/Freemail/src/freemail/InboundContact.java
> ===================================================================
> --- trunk/apps/Freemail/src/freemail/InboundContact.java	2008-05-20 16:30:25 
UTC (rev 19975)
> +++ trunk/apps/Freemail/src/freemail/InboundContact.java	2008-05-20 20:24:40 
UTC (rev 19976)
> @@ -58,7 +58,7 @@
>  			this.ibct_dir.mkdir();
>  		}
>  		
> -		this.ibct_props = new PropsFile(new File(this.ibct_dir, IBCT_PROPSFILE));
> +		this.ibct_props = PropsFile.createPropsFile(new File(this.ibct_dir, 
IBCT_PROPSFILE));
>  	}
>  	
>  	public void setProp(String key, String val) {
> @@ -115,7 +115,7 @@
>  			Logger.normal(this,"Found a message!");
>  			
>  			// parse the Freemail header(s) out.
> -			PropsFile msgprops = new PropsFile(msg, true);
> +			PropsFile msgprops = PropsFile.createPropsFile(msg, true);
>  			String s_id = msgprops.get("id");
>  			if (s_id == null) {
>  				Logger.error(this,"Got a message with an invalid header. Discarding.");
> @@ -199,7 +199,7 @@
>  	public boolean validateFrom(EmailAddress from) throws IOException, 
ConnectionTerminatedException {
>  		String sd = from.getSubDomain();
>  		if (sd == null) {
> -			// well that's definately not valid. Piffle!
> +			// well that's definitely not valid. Piffle!
>  			return false;
>  		}
>  		
> 
> Modified: trunk/apps/Freemail/src/freemail/OutboundContact.java
> ===================================================================
> --- trunk/apps/Freemail/src/freemail/OutboundContact.java	2008-05-20 
16:30:25 UTC (rev 19975)
> +++ trunk/apps/Freemail/src/freemail/OutboundContact.java	2008-05-20 
20:24:40 UTC (rev 19976)
> @@ -139,7 +139,7 @@
>  				throw new IOException("Couldn't create outbound contact dir!");
>  			}
>  			
> -			this.contactfile = new PropsFile(new File(obctdir, PROPSFILE_NAME));
> +			this.contactfile = PropsFile.createPropsFile(new File(obctdir, 
PROPSFILE_NAME));
>  			this.ctoutbox = new File(obctdir, OUTBOX_DIR);
>  			if (!this.ctoutbox.exists() && !this.ctoutbox.mkdir()) {
>  				throw new IOException("Couldn't create contact outbox!");
> @@ -152,7 +152,7 @@
>  		this.address = new EmailAddress();
>  		this.address.domain = ctdir.getName()+".freemail";
>  		
> -		this.contactfile = new PropsFile(new File(ctdir, PROPSFILE_NAME));
> +		this.contactfile = PropsFile.createPropsFile(new File(ctdir, 
PROPSFILE_NAME));
>  		
>  		this.ctoutbox = new File(ctdir, OUTBOX_DIR);
>  		if (!this.ctoutbox.exists()) {
> @@ -540,7 +540,7 @@
>  		
>  		Logger.normal(this,"got mailsite");
>  		
> -		PropsFile mailsite = new PropsFile(mailsite_file);
> +		PropsFile mailsite = PropsFile.createPropsFile(mailsite_file);
>  		
>  		String rtsksk = mailsite.get("rtsksk");
>  		String keymod_str = mailsite.get("asymkey.modulus");
> @@ -834,7 +834,7 @@
>  			this.uid = uid;
>  			this.file = new File(ctoutbox, Integer.toString(uid));
>  			
> -			this.index = new PropsFile(new File(ctoutbox, INDEX_FILE));
> +			this.index = PropsFile.createPropsFile(file);
>  			
>  			this.slot = this.index.get(uid+".slot");
>  			String s_first = this.index.get(uid+".first_send_time");
> 
> Modified: trunk/apps/Freemail/src/freemail/RTSFetcher.java
> ===================================================================
> --- trunk/apps/Freemail/src/freemail/RTSFetcher.java	2008-05-20 16:30:25 UTC 
(rev 19975)
> +++ trunk/apps/Freemail/src/freemail/RTSFetcher.java	2008-05-20 20:24:40 UTC 
(rev 19976)
> @@ -288,7 +288,7 @@
>  			return false;
>  		}
>  		
> -		PropsFile rtsprops = new PropsFile(rtsfile);
> +		PropsFile rtsprops = PropsFile.createPropsFile(rtsfile);
>  		
>  		try {
>  			validate_rts(rtsprops);
> @@ -335,7 +335,7 @@
>  			return false;
>  		}
>  		
> -		PropsFile mailsite = new PropsFile(msfile);
> +		PropsFile mailsite = PropsFile.createPropsFile(msfile);
>  		String their_exponent = mailsite.get("asymkey.pubexponent");
>  		String their_modulus = mailsite.get("asymkey.modulus");
>  		
> 
> Modified: trunk/apps/Freemail/src/freemail/RTSLog.java
> ===================================================================
> --- trunk/apps/Freemail/src/freemail/RTSLog.java	2008-05-20 16:30:25 UTC 
(rev 19975)
> +++ trunk/apps/Freemail/src/freemail/RTSLog.java	2008-05-20 20:24:40 UTC 
(rev 19976)
> @@ -39,7 +39,7 @@
>  	private static String UNPROC_NEXTID = "unproc-nextid";
>  
>  	public RTSLog(File f) {
> -		this.logfile = new PropsFile(f);
> +		this.logfile = PropsFile.createPropsFile(f);
>  		if (!this.logfile.exists()) {
>  			String birth = DateStringFactory.getOffsetKeyString(0);
>  			this.logfile.put("birth", birth);
> 
> Modified: trunk/apps/Freemail/src/freemail/config/Configurator.java
> ===================================================================
> --- trunk/apps/Freemail/src/freemail/config/Configurator.java	2008-05-20 
16:30:25 UTC (rev 19975)
> +++ trunk/apps/Freemail/src/freemail/config/Configurator.java	2008-05-20 
20:24:40 UTC (rev 19976)
> @@ -35,7 +35,7 @@
>  	private final HashMap callbacks;
>  
>  	public Configurator(File f) {
> -		this.props = new PropsFile(f);
> +		this.props = PropsFile.createPropsFile(f);
>  		this.props.setCommentPrefix("#");
>  		String ls = System.getProperty("line.separator");
>  		StringBuffer head = new StringBuffer();
> 
> Modified: trunk/apps/Freemail/src/freemail/utils/PropsFile.java
> ===================================================================
> --- trunk/apps/Freemail/src/freemail/utils/PropsFile.java	2008-05-20 
16:30:25 UTC (rev 19975)
> +++ trunk/apps/Freemail/src/freemail/utils/PropsFile.java	2008-05-20 
20:24:40 UTC (rev 19976)
> @@ -31,8 +31,31 @@
>  import java.util.Iterator;
>  import java.util.Map;
>  import java.util.Set;
> +import java.util.Hashtable;
>  
>  public class PropsFile {
> +	// substitute static methods for constructor
> +	
> +	static Hashtable propsList=new Hashtable();
> +	
> +	public static PropsFile createPropsFile(File f, boolean stopAtBlank) {
> +		String fn=f.getPath();
> +
> +		PropsFile pf=(PropsFile)propsList.get(fn);
> +		
> +		if(pf!=null) {
> +			return pf;
> +		} else {
> +			pf=new PropsFile(f, stopAtBlank);
> +			propsList.put(fn, pf);
> +			return pf;
> +		}
> +	}
> +
> +	public static PropsFile createPropsFile(File f) {
> +		return createPropsFile(f, false);
> +	}
> +
>  	private final File file;
>  	private HashMap data;
>  	private BufferedReader bufrdr;
> @@ -43,7 +66,7 @@
>  	 * a blank line. It's the the caller's responsibility to get
>  	 * (using the getReader() method) the stream and close it properly.
>  	 */
> -	public PropsFile(File f, boolean stopAtBlank) {
> +	private PropsFile(File f, boolean stopAtBlank) {
>  		this.file = f;
>  		this.data = null;
>  		
> @@ -57,10 +80,6 @@
>  		this.header = null;
>  	}
>  	
> -	public PropsFile(File f) {
> -		this(f, false);
> -	}
> -	
>  	public void setCommentPrefix(String cp) {
>  		this.commentPrefix = cp;
>  	}
> 
> _______________________________________________
> Freemail mailing list
> Freemail at freenetproject.org
> http://emu.freenetproject.org/cgi-bin/mailman/listinfo/freemail
>