bgware: Thread: mailfront imapfront-auth with dovecot 1.2.16 : SUCCESS

Hi,

This is just to share with the list members that upon couple hours of 
researching, I have successfully run mailfront v1.17 imapfront-auth with 
dovecot v1.2.16 which doesn't require dovecot service to be running.

I know http://mij.oltrelinux.com/net/dovecot-qmail-vmailmgr/ and that 
requires dovecot service running and I can't use with imapfront-auth.  I 
prefer to use mailfront + cvm (pop3d/pop3sd with pop3front-auth and 
imapd/imapsd with imapfront-auth) so that all related mail services are 
running under daemontools + ucspi-tcp.

Below is the wrapper /usr/bin/mailfront-imapfront-auth-dovecot or name 
whatever you like and chmod +x /usr/bin/mailfront-imapfront-auth-dovecot:
--------------------8<----------8<----------8<----------8<---------------------
#!/bin/sh

if [ -n "$MAIL" ] ; then
	export MAIL=maildir:`echo $MAIL | sed 's@:@::@g'`
fi
exec "$@"
--------------------8<----------8<----------8<----------8<---------------------

Below is example imapd/run file which works for courier-imap and dovecot:
--------------------8<----------8<----------8<----------8<---------------------
#!/bin/sh

DOVECOTWRAPPER=""
# Default to support dovecot 1.0.x
if [ -x "/usr/libexec/dovecot/imap" ] ; then
	IMAPD=/usr/libexec/dovecot/imap
	if [ -x "/usr/bin/mailfront-imapfront-auth-dovecot" ] ; then
		DOVECOTWRAPPER=/usr/bin/mailfront-imapfront-auth-dovecot
	elif [ -x "/usr/local/bin/mailfront-imapfront-auth-dovecot" ] ; then
		DOVECOTWRAPPER=/usr/local/bin/mailfront-imapfront-auth-dovecot
	fi
elif [ -x "/usr/local/bin/imapd" ] ; then
	IMAPD=/usr/local/bin/imapd
elif [ -x "/usr/lib/courier-imap/bin/imapd" ] ; then
	IMAPD=/usr/lib/courier-imap/bin/imapd
else
	IMAPD=/usr/bin/imapd
fi

if [ -x "/usr/bin/imapfront-auth" ] ; then
	IMAPFRONT_AUTH=/usr/bin/imapfront-auth
else
	IMAPFRONT_AUTH=/usr/local/bin/imapfront-auth
fi

ulimitdata=500000000
concurrency=40

CVM_SASL_PLAIN=cvm-local:/tmp/.cvm-vmailmgr
CVM_SASL_LOGIN=cvm-local:/tmp/.cvm-vmailmgr
export CVM_SASL_PLAIN
export CVM_SASL_LOGIN

exec \
softlimit -m $ulimitdata \
tcpserver -DHRUvX -c "$concurrency" -l "`head -n 1 /var/qmail/control/me`" \
	-x /etc/tcpcontrol/imap.cdb 0 imap \
$IMAPFRONT_AUTH $DOVECOTWRAPPER \
$IMAPD 2>&1
--------------------8<----------8<----------8<----------8<---------------------

C wrapper source can be found at:

http://choon.net/mailfront-imapfront-auth-dovecot.php

Many thanks to MIJ at http://mij.oltrelinux.com as I copied one of his 
source code function.  Sorry, I am not good in C at all and only code in 
C occasionally.

Hope the above information is useful to someone ;)

Thanks.

Kindest regards,
Giam Teck Choon

On Wed, Nov 10, 2010 at 09:20:58AM +0800, Giam Teck Choon wrote:
> This is just to share with the list members that upon couple hours of 
> researching, I have successfully run mailfront v1.17 imapfront-auth with 
> dovecot v1.2.16 which doesn't require dovecot service to be running.
> 
> Below is the wrapper /usr/bin/mailfront-imapfront-auth-dovecot or name 
> whatever you like and chmod +x /usr/bin/mailfront-imapfront-auth-dovecot:
> --------------------8<----------8<----------8<----------8<---------------------
> #!/bin/sh
> 
> if [ -n "$MAIL" ] ; then
> 	export MAIL=maildir:`echo $MAIL | sed 's@:@::@g'`
> fi
> exec "$@"

Well, if that's all that's needed, I can add that right into
imapfront-auth itself.

Please try the appended patch.  Simply set SETUP_DOVECOT=1 (or =
anything) before running imapfront-auth, and it should set $MAIL to an
appropriate string.  I am working on testing it myself, but the test CVM
I am currently using doesn't actually set a mail fact.

-- 
Bruce Guenter ####@####.####                http://untroubled.org/

diff --git a/imapfront-auth.c b/imapfront-auth.c
index 3f85d21..eecb658 100644
--- a/imapfront-auth.c
+++ b/imapfront-auth.c
@@ -245,15 +245,37 @@ void cmd_capability(void)
   respond(0, "OK CAPABILITY completed");
 }
 
+static int setup_env(void)
+{
+  const char* maildir;
+  const char* colon;
+
+  if (cvm_fact_mailbox != 0
+      && getenv("SETUP_DOVECOT")) {
+    /* Use cmd for temporary storage of the substituted mailbox name */
+    if (!str_copys(&cmd, "maildir:"))
+      return 0;
+    maildir = cvm_fact_mailbox;
+    while ((colon = strchr(maildir, ':')) != 0) {
+      if (!str_catb(&cmd, maildir, colon-maildir)
+	  || !str_catb(&cmd, "::", 2))
+	return 0;
+      maildir = colon + 1;
+    }
+    if (!str_cats(&cmd, maildir))
+      return 0;
+    cvm_fact_mailbox = cmd.s;
+  }
+  return cvm_setenv()
+    && setenv("IMAPLOGINTAG", tag.s, 1) == 0
+    && setenv("AUTHENTICATED", cvm_fact_username, 1) == 0;
+}
+
 void do_exec(void)
 {
   if (!cvm_setugid())
     respond(0, "NO Internal error: could not set UID/GID");
-  else if (!cvm_setenv() ||
-	   (cvm_fact_mailbox != 0 &&
-	    setenv("MAILDIR", cvm_fact_mailbox, 1) == -1) ||
-	   setenv("IMAPLOGINTAG", tag.s, 1) == -1 ||
-	   setenv("AUTHENTICATED", cvm_fact_username, 1) == -1)
+  else if (!setup_env())
     respond(0, "NO Internal error: could not set environment");
   else {
     alarm(0);

On 11/11/2010 04:07 AM, Bruce Guenter wrote:
> On Wed, Nov 10, 2010 at 09:20:58AM +0800, Giam Teck Choon wrote:
>> This is just to share with the list members that upon couple hours of
>> researching, I have successfully run mailfront v1.17 imapfront-auth with
>> dovecot v1.2.16 which doesn't require dovecot service to be running.
>>
>> Below is the wrapper /usr/bin/mailfront-imapfront-auth-dovecot or name
>> whatever you like and chmod +x /usr/bin/mailfront-imapfront-auth-dovecot:
>> --------------------8<----------8<----------8<----------8<---------------------
>> #!/bin/sh
>>
>> if [ -n "$MAIL" ] ; then
>> 	export MAIL=maildir:`echo $MAIL | sed 's@:@::@g'`
>> fi
>> exec "$@"
>
> Well, if that's all that's needed, I can add that right into
> imapfront-auth itself.
>
> Please try the appended patch.  Simply set SETUP_DOVECOT=1 (or =
> anything) before running imapfront-auth, and it should set $MAIL to an
> appropriate string.  I am working on testing it myself, but the test CVM
> I am currently using doesn't actually set a mail fact.
>

Yes, the patch works!  I should ask you first before spending time to 
code the wrapper :p

Below is the imapd/run file:

--------------------8<----------8<----------8<----------8<---------------------
#!/bin/sh

# Default to support dovecot
if [ -x "/usr/libexec/dovecot/imap" ] ; then
     IMAPD=/usr/libexec/dovecot/imap
     export SETUP_DOVECOT=1
elif [ -x "/usr/local/bin/imapd" ] ; then
     IMAPD=/usr/local/bin/imapd
elif [ -x "/usr/lib/courier-imap/bin/imapd" ] ; then
     IMAPD=/usr/lib/courier-imap/bin/imapd
else
     IMAPD=/usr/bin/imapd
fi

if [ -x "/usr/bin/imapfront-auth" ] ; then
     IMAPFRONT_AUTH=/usr/bin/imapfront-auth
else
     IMAPFRONT_AUTH=/usr/local/bin/imapfront-auth
fi

ulimitdata=500000000
concurrency=40

CVM_SASL_PLAIN=cvm-local:/tmp/.cvm-vmailmgr
CVM_SASL_LOGIN=cvm-local:/tmp/.cvm-vmailmgr
export CVM_SASL_PLAIN
export CVM_SASL_LOGIN

exec \
softlimit -m $ulimitdata \
tcpserver -DHRUvX -c "$concurrency" -l "`head -n 1 /var/qmail/control/me`" \
     -x /etc/tcpcontrol/imap.cdb 0 imap \
$IMAPFRONT_AUTH \
$IMAPD 2>&1
--------------------8<----------8<----------8<----------8<---------------------

Thanks.

Kindest regards,
Giam Teck Choon

On Thu, Nov 11, 2010 at 05:38:43AM +0800, Giam Teck Choon wrote:
> > Please try the appended patch.  Simply set SETUP_DOVECOT=1 (or =
> > anything) before running imapfront-auth, and it should set $MAIL to an
> > appropriate string.
> 
> Yes, the patch works!

Excellent.  Is the choice of $SETUP_DOVECOT a reasonable name, or should
I use something else (like $DOVECOT_MAIL etc)?

I see by the wiki that this works for version 2 as well, and also
supports mbox files.  I will add a stat to the imapfront-auth code to
handle both cases.

-- 
Bruce Guenter ####@####.####                http://untroubled.org/

On 11/11/2010 06:21 AM, Bruce Guenter wrote:
> On Thu, Nov 11, 2010 at 05:38:43AM +0800, Giam Teck Choon wrote:
>>> Please try the appended patch.  Simply set SETUP_DOVECOT=1 (or =
>>> anything) before running imapfront-auth, and it should set $MAIL to an
>>> appropriate string.
>>
>> Yes, the patch works!
>
> Excellent.  Is the choice of $SETUP_DOVECOT a reasonable name, or should
> I use something else (like $DOVECOT_MAIL etc)?

If you ask me about appropriate name, I think $DOVECOT_MAIL_LOCATION or 
$DOVECOT_MAILLOCATION since we are passing the MailLocation to dovecot 
imap directly.  Anyway, whatever name you like doesn't matter as long as 
it works :)

I think if your imapfront-auth can handle dovecot different 
mail_location format would be great.  Yes, read your comment about 
adding stat to handle both cases.... ... :)  So how you stat the two 
format?  Just curious.  By using the $SETUP_DOVECOT variable?  Like 
SETUP_DOVECOT=mbox|maildir?  Or simply stat the $MAIL to see it is a 
directory or a file to differential maildir or mbox respectively before 
modifying $MAIL to maildir or mbox etc. to dovecot imap?

http://wiki.dovecot.org/MailLocation

mbox and maildir are well know standards I guess but dovecot has their 
own called dbox (http://wiki.dovecot.org/MailboxFormat/dbox).  Do you 
plan to support such?  Not a requirement since making mailfront + cvm 
natively support dovecot is primary goal without excessive 
hacking/patching will benefit other users :)


>
> I see by the wiki that this works for version 2 as well, and also
> supports mbox files.  I will add a stat to the imapfront-auth code to
> handle both cases.
>

That is my next task to test for version 2 actually :p

If their version 2 are using $MAIL for mail_location (same as their 
version 1 which I guess should be) and with double colons "::" escape 
feature in their version 2 as well, I don't see why won't work :p

If you are interested, I can report back my progress about the test for 
their version 2?

Many thanks for your prompt response!

Kindest regards,
Giam Teck Choon

On Thu, Nov 11, 2010 at 07:04:08AM +0800, Giam Teck Choon wrote:
>  So how you stat the two 
> format?  Just curious.  By using the $SETUP_DOVECOT variable?  Like 
> SETUP_DOVECOT=mbox|maildir?  Or simply stat the $MAIL to see it is a 
> directory or a file to differential maildir or mbox respectively before 
> modifying $MAIL to maildir or mbox etc. to dovecot imap?

That's exactly what I've done now.  Directory is assumed to be maildir,
anything else (including missing file) is assumed to be mbox.

> mbox and maildir are well know standards I guess but dovecot has their 
> own called dbox (http://wiki.dovecot.org/MailboxFormat/dbox).  Do you 
> plan to support such?

Not planned, no, though I could be persuaded.

> If you are interested, I can report back my progress about the test for 
> their version 2?

Definitely, please do.

-- 
Bruce Guenter ####@####.####                http://untroubled.org/

On 11/11/2010 08:33 AM, Bruce Guenter wrote:
> On Thu, Nov 11, 2010 at 07:04:08AM +0800, Giam Teck Choon wrote:
>>   So how you stat the two
>> format?  Just curious.  By using the $SETUP_DOVECOT variable?  Like
>> SETUP_DOVECOT=mbox|maildir?  Or simply stat the $MAIL to see it is a
>> directory or a file to differential maildir or mbox respectively before
>> modifying $MAIL to maildir or mbox etc. to dovecot imap?
>
> That's exactly what I've done now.  Directory is assumed to be maildir,
> anything else (including missing file) is assumed to be mbox.

Ok.

>
>> mbox and maildir are well know standards I guess but dovecot has their
>> own called dbox (http://wiki.dovecot.org/MailboxFormat/dbox).  Do you
>> plan to support such?
>
> Not planned, no, though I could be persuaded.
>
>> If you are interested, I can report back my progress about the test for
>> their version 2?
>
> Definitely, please do.
>

Things not what we expected.  Their version 2.0.7 not honouring $MAIL at 
all.  I will keep on testing various environment variables and report 
back if I find out more.

Thanks.

Kindest regards,
Giam Teck Choon

SUCCESS with v2.0.7... Many thanks to Timo from dovecot.org :)

> Things not what we expected. Their version 2.0.7 not honouring $MAIL at
> all. I will keep on testing various environment variables and report
> back if I find out more.

Wrapper below for dovecot v2.0.7

--------------------8<----------8<----------8<----------8<---------------------
#!/bin/sh

if [ -n "$MAIL" ] ; then
	export MAIL=maildir:`echo $MAIL | sed 's@:@::@g'`
fi
exec "$@" -O -o mail_location=$MAIL
--------------------8<----------8<----------8<----------8<---------------------

Thanks.

Kindest regards,
Giam Teck Choon

On Fri, Nov 12, 2010 at 03:08:24AM +0800, Giam Teck Choon wrote:
> SUCCESS with v2.0.7... Many thanks to Timo from dovecot.org :)
> > Things not what we expected. Their version 2.0.7 not honouring $MAIL at
> > all. I will keep on testing various environment variables and report
> > back if I find out more.
> 
> exec "$@" -O -o mail_location=$MAIL

So you could replace the $IMAPD part of the original script with:

	sh -c "exec $IMAPD -O -o mail_location=\$MAIL"

so it would end up running:

	tcpserver [options] \
	imapfront-auth \
	sh -c 'exec /usr/libexec/dovecot/imap  -O -o mail_location=$MAIL'

making sh do the substitution of $MAIL.  No wrapper script should be
needed for that.

-- 
Bruce Guenter ####@####.####                http://untroubled.org/

On 11/12/2010 03:56 AM, Bruce Guenter wrote:
> On Fri, Nov 12, 2010 at 03:08:24AM +0800, Giam Teck Choon wrote:
>> SUCCESS with v2.0.7... Many thanks to Timo from dovecot.org :)
>>> Things not what we expected. Their version 2.0.7 not honouring $MAIL at
>>> all. I will keep on testing various environment variables and report
>>> back if I find out more.
>>
>> exec "$@" -O -o mail_location=$MAIL
>
> So you could replace the $IMAPD part of the original script with:
>
> 	sh -c "exec $IMAPD -O -o mail_location=\$MAIL"
>
> so it would end up running:
>
> 	tcpserver [options] \
> 	imapfront-auth \
> 	sh -c 'exec /usr/libexec/dovecot/imap  -O -o mail_location=$MAIL'
>
> making sh do the substitution of $MAIL.  No wrapper script should be
> needed for that.
>

If v1.2.16 not support the options?  I haven't test that in depth but 
mostly will not.  I asked Timo and still waiting for reply.  If v1 not 
supporting those options, a wrapper is needed.

On my choon.net server:
# /usr/libexec/dovecot/imap -O -o mail_location=maildir:/tmp/haha
* PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID 
ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND 
UNSELECT IDLE CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 
CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH 
LIST-STATUS] Logged in as root
* BYE Server shutting down.
Info: Server shutting down. bytes=0/332
# /usr/sbin/dovecot --version
2.0.7

On my client server:
# /usr/libexec/dovecot/imap -O -o mail_location=maildir:/tmp/haha
imap(root): Error: Ambiguous mail location setting, don't know what to 
do with it: /root/Maildir/ (try prefixing it with mbox: or maildir:)
imap(root): Fatal: Mail storage creation failed with mail_location: 
/root/Maildir/
# /usr/sbin/dovecot --version
1.0.15

So I guess a wrapper is needed.  I need a generic imapd/run script which 
support both dovecot versions (v1 and v2) and courier-imap.  If just for 
a specific imap server like dovecot v2 then wrapper not needed like what 
you stated.

Wrapper as below to support dovecot v1 and v2:
--------------------8<----------8<----------8<----------8<---------------------
#!/bin/sh

if [ -n "$MAIL" ] ; then
	export MAIL=maildir:`echo $MAIL | sed 's@:@::@g'`
fi
# we need to know dovecot is v1 or v2
# as v1 might not be supporting the options
# -O -o mail_location=... ...
DOVECOTIMAPOPTIONS=""
if [ -x "/usr/sbin/dovecot" ] ; then
     dovecotversion=`/usr/sbin/dovecot --version | cut -f1 -d '.'`
     if [ "x$dovecotversion" == "x2" ] ; then
         DOVECOTIMAPOPTIONS="-O -o mail_location=$MAIL"
     fi
fi
exec "$@" $DOVECOTIMAPOPTIONS
--------------------8<----------8<----------8<----------8<---------------------

Thanks.

Kindest regards,
Giam Teck Choon