Fix the things

Backup IMAP with Horde Webmail, or Outlook or Thunderbird Email Client

Overview

Overview
Horde
Imap Configuration for thunderbird
Thunderbird Plugin
Thunderbird backup creation
Thunderbird backup import
Microsoft Outlook

Backing up email locally onto your computer can be achieved through creating a backup of said email accounts through the email clients. Horde webmail allows a simple way of doing this, but you must have a client that accepts .mbox files. Thunderbird is a free email client, offered by mozilla, that has a plugin available which allows the use of .mbox files.

Outlook cannot read mbox files. Thunderbird and horde webmail both work off of mbox exports. There are third party applications, which cost money, that have pst to mbox file converter capabilities.

Horde

Horde Webmail makes it simple to create a mbox file.

First login to webmail. Your webmail login URL info might not be webmail.domain.com but the example shown here is. Usually domain.com:2096 will be webmail. So for instance disc4life.com:2096 also works in this example.

Second select Read Mail Using Horde.

Third right click on your Inbox and Select Export.

Fourth download and Save the file as INBOX.mbox.

Now that you have the mbox file downloaded you can upload / import to another webmail account or import through an email client. You have to do this for each folder that has email you wish to keep from the account.

Importing through webmail is done by selecting the file after right clicking inbox and selecting import in the horde mail client.

Imap Configuration for thunderbird

If you want you can configure your thunderbird client to connect to your account and generate the mbox file that way instead. We also need the plugin to import the mbox file.

Thunderbird Email client downloaded and configure for IMAP, not pop3.

For example my email is disc4life@disc4life.com and my email server incoming and outgoing is disc4life.com. I use SSL settings for IMAP port 993 and SMTP port 465. Authentication Required Normal Password Authorization.

You need the correct settings, IE account name, password, incoming and outgoing server, ports, authentication method for this to work on your server / account.

Thunderbird Plugin

If you are just storing your email locally in thunderbird, and you have download the mbox file through webmail, then you do not need to configure your account as shown above in Imap configuration.

First download the Import / Export Tools add-on. https://addons.mozilla.org/en-us/thunderbird/addon/importexporttools/ This allows you to archive email accounts through Thunderbird. Thunderbird does have archiving but that just works server side and not client side. You need this add-on to archive your email locally. This also allows use of mbox files.

1. Once the tool is downloaded and you have Thunderbird open Click the Menu Button, indicated by the red arrow, and select Add-ons.

2.Goto Extensions and Click the Gearbox then Select Install Add-on from file…

3.Navigate to the .xpi file you just downloaded and click Open, Select Install now on the next screen. Once the installation is complete restart Thunder Bird.

If you exported with webmail skip to #7 or importing.

Thunderbird backup import

Thunderbird backup creation

Exporting:

4.Select the Inbox / Folder you want to backup. Right click it and Select Export folder (zipped)

5. Select your backup destination. It will have the same name as the folder you are backing up and it will backup to your currently selected folder.

6.Navigate to the file and right click it, then Select Extract All… On the screen that pops up Extract to the current location.

Thunderbird backup import

Importing:

7. Go back to Thunderbird and right click on Local Folders. Select New Folder. Name that folder what ever you please, I choose BackedupEmail. Click Create Folder.

8. Right click your newly created folder and navigate ImportExportTools and select Import mbox file.

9. Leave the default selection of “Import directly one or more mbox files. Click Ok.

10. Navigate to your Extracted file, click it, and click Open.

11. You can now view your emails under your Local Folders. Just double click on the BackedupEmail Folder and it should expand to show your Inbox that you just imported.

Overview

Microsoft Outlook

Outlook Instructions:

1. Click on File on the in the upper left hand Corner.

2. Select Open & Export. If you do not have that option just click Import / Export

3. Select Export To a File, Click Next. Choose Outlook Data File (.pst), Click Next.

4. Select your account, this one happens to be disc4life@disc4life.com, Click Next.

5. Choose a backup name, this one is disc4lifebackup.pst. Click Finish. It will bring up a password prompt. You can leave it blank or insert a password. I left mine blank.

Congratulations you have just exported your account in Outlook! Now it is time to import locally.

Importing:

6. Now Open the Import / Export once again by Selecting File -> Open & Export -> Import / Export. Select Import From another program or file, Click Next.

7. Select Outlook Data File (.pst), Click Next.

8. Either Input or Browse to the .pst file, this one is disc4lifebackup.pst, Click Next

9. Select Outlook Data File, Click Finish. You can now view your emails under Outlook Data File. This means these emails are now local and not hosted on the server. You can now clean up your inbox on the server and be sure that your emails now have a recent backup.

Top

MySQL Configuration

I’ve been screwing up mysql configurations left and right. Nothing really harmful just not efficient. Not enough ram allocated to the right configurations.

http://www.mysqlperformanceblog.com/2014/01/28/10-mysql-settings-to-tune-after-installation/
http://mysql.rjweb.org/doc.php/memory

Two great resources for reading about how to properly configure mysql.

Checking how much ram you need simple. You just need a few mysql commands to get you the numbers.

SELECT ENGINE,

ROUND(SUM(data_length) /1024/1024, 1) AS “Data MB”,

ROUND(SUM(index_length)/1024/1024, 1) AS “Index MB”,

ROUND(SUM(data_length + index_length)/1024/1024, 1) AS “Total MB”,

COUNT(*) “Num Tables”

FROM INFORMATION_SCHEMA.TABLES

WHERE table_schema not in (“information_schema”, “performance_schema”)

GROUP BY ENGINE;

(this command wont work pasted in as it is posted here, its right but wordpress formats it incorrectly, get the command from the link I posted above)

Thats the most imporant. This command shows you what you should be setting your two more important settings to. .

“Add up Data_length + Index_length for all the InnoDB tables.Set innodb_buffer_pool_size to no more than 110% of that total.

Add up Index_length for all the MyISAM tables.Set key_buffer_size no larger than that size.”

These settings control how the engines will interact with the ram and hard disk. Innodb needs more ram than my ISAM. So if you have 200mb of myisam index and 200 total mb of innodb then they would require similar amounts of ram. This would probably mean myisam is doing more total work though. Its important to know which engine is being used more.

”
SHOW GLOBAL STATUS LIKE ‘Key%’;
then calculate Key_read_requests / Key_reads If it is high (say, over 10), then the key_buffer is big enough.
”
”
If using just MyISAM, set key_buffer_size to 20% of _available_ RAM.(Plus innodb_buffer_pool_size=0)

If using just InnoDB, set innodb_buffer_pool_size to 70% of _available_ RAM.(Plus key_buffer_size = 10M, small, but not zero.)
“

This is for dedicated MySQL servers. If you have a webserver running together with MySQL in one box then consider lowering these values. Thats why you check to see how large the tables are for each engine so you can set the buffers accordingly.

Query Cache
Short answer: query_cache_type = OFF and query_cache_size = 0
To see how well your QC is performing,SHOW GLOBAL STATUS LIKE ‘Qc%’;then compute the read hit rate:Qcache_hits / Qcache_insertsIf it is over, say, 5, the QC might be worth keeping.ENGINE=MEMORY

This is a little-used alternative to MyISAM and InnoDB. It has limited uses. The size of a MEMORY table is limited to max_heap_table_size, which defaults to 16MB. I mention it in case you have changed the value to something huge; this would stealing from other possible uses of RAM.

You can see how well your system is performing via
SHOW GLOBAL STATUS;
or
SHOW /*!50002 GLOBAL */ STATUS;

and computing the opens/second via
Opened_files / Uptime If this is more than, say, 5, table_cache should be increased.If it is less than, say, 1, you might get improvement by decreasing table_cache.

InnoDB only: innodb_buffer_pool_size = 70% of RAM If you have lots of RAM and are using 5.5 (or later), then considerhaving multiple pools.Recommend 1-16 innodb_buffer_pool_instances, such that each one is no smaller than 1GB.(Sorry, no metric on how much this will help; probably not a lot.)

Meanwhile, set key_buffer_size = 20M (tiny, but non-zero)
If you have a mixture of engines, lower both numbers.

Alot of this is copy pasta from those articles. You should read those articles if you want to understand this configuration for MySQL.

max_allowed_packet=2M

This setting is like php memory_limit. It is the limit on each query. If you start seeing out of memory errors from the mysql error log then you should raise this value. Otherwise 2M is probably plenty and most querys will not be this large.

wait_timeout=12

This is how long the connection will stay open after a query has been received. It usually takes a second or two to send a query.
“The client had been sleeping more than wait_timeout or interactive_timeout seconds without issuing any requests to the server. See Section 5.1.4, “Server System Variables”. ”
12 is probably too high but its better than 6. If you’re having issues reaching max connections maybe try 8 or 10.
Having it too low could cause Aborted_clients. But there are plenty of other reasons a client could abort.

http://dev.mysql.com/doc/refman/5.0/en/communication-errors.html

” Other reasons for problems with aborted clients or aborted connections:

The max_allowed_packet variable value is too small or queries require more memory than you have allocated for mysqld. See Section B.5.2.10, “Packet Too Large”.

If you start the server with the –log-warnings option, you might find messages like this in your error log:
If a client successfully connects but later disconnects improperly or is terminated, the server increments the Aborted_clients status variable, and logs an Aborted connection message to the error log.”

innodb_file_per_table=1
skip-external-locking

http://doc.ctrlaltdel.ch/database/mysql/manual_Performance.html

So stop messing up those configurations already jeeeeeeezzzzz.

Well I figured it out!

GTmetrix analyzes your website and you can see how many requests are being processed through GET commands to the web server. I just had to edit my .htaccess file. The inital load has 11 requests but the subsequent ones now have much less because gzip compression and mainly browser caching have been added to the .htaccess file. This lets the client computer store data that would normally be taken from the server on the initial load of the page. Once its loaded on the clients computer it no longer has to request that same GET command and reduces load on the server and the clients connection.

And here is what you add to that .htaccess file to enable these features for your webpage.

#compress text, HTML, JavaScript, CSS, and XML
AddOutputFilterByType DEFLATE text/plain
AddOutputFilterByType DEFLATE text/html
AddOutputFilterByType DEFLATE text/xml
AddOutputFilterByType DEFLATE text/css
AddOutputFilterByType DEFLATE application/xml
AddOutputFilterByType DEFLATE application/xhtml+xml
AddOutputFilterByType DEFLATE application/rss+xml
AddOutputFilterByType DEFLATE application/javascript
AddOutputFilterByType DEFLATE application/x-javascript

# remove browser bugs
BrowserMatch ^Mozilla/4 gzip-only-text/html
BrowserMatch ^Mozilla/4\.0[678] no-gzip
BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
Header append Vary User-Agent

## EXPIRES CACHING ##

ExpiresActive On
ExpiresByType image/jpg “access plus 1 year”
ExpiresByType image/jpeg “access plus 1 year”
ExpiresByType image/gif “access plus 1 year”
ExpiresByType image/png “access plus 1 year”
ExpiresByType text/css “access plus 1 month”
ExpiresByType application/pdf “access plus 1 month”
ExpiresByType text/x-javascript “access plus 1 month”
ExpiresByType application/x-shockwave-flash “access plus 1 month”
ExpiresByType image/x-icon “access plus 1 year”
ExpiresDefault “access plus 7 days”

## EXPIRES CACHING ##

Boom its that easy. Save that file and your good to go.

Another note is GTmetrix can sometimes have false positives. So my site grades a 97% pagespeed anda 96% Yslow but it says I could be doing more caching. This isnt true because google fonts are already cached. I can’t control the header of the google fonts but the header already has caching. I’m sure there are other false positives. Also I am no coder so the content “Remove query strings from static resources” will continue to have a low score. I’ll figure out how to edit that eventually. For now I’m pretty happy with the overall results. Now after the initial load I only see 1 GET request. Just as I thought it should have been to begin with.

How do you configure that?

Before clicking on this page I had 0 connections open to my server.

root@host [~]# netstat -plan|grep :80|awk {‘print $5’}|cut -d: -f 1|sort|uniq -c|sort -nk 1
1
1 0.0.0.0
1 127.0.0.1

Now I have 20.

root@host [~]# netstat -plan|grep :80|awk {‘print $5’}|cut -d: -f 1|sort|uniq -c|sort -nk 1
1
1 0.0.0.0
1 69.16.208.35
20 23.28.18.128

I opened the window in another browser and I have 42 connections now…

Why does clicking on two pages open 42 different connections to port 80 from my modem. Does this mean that 42 requests were made in Apache? I don’t know but I would like to know how to configure this. Seems silly that it would need more than one connection per a page served.