{"id":872,"date":"2020-09-21T15:41:10","date_gmt":"2020-09-21T22:41:10","guid":{"rendered":"http:\/\/10.0.1.201\/?p=872"},"modified":"2025-12-01T11:33:12","modified_gmt":"2025-12-01T18:33:12","slug":"parsing-e-mail-for-info","status":"publish","type":"post","link":"https:\/\/strawhousepig.net\/wordpress\/2020\/09\/21\/parsing-e-mail-for-info\/","title":{"rendered":"Parsing e-mail for info"},"content":{"rendered":"\n

[UPDATE] Added a version that works on OS X 10.4 to the bottom of this post. Does some different logging and is tighter, so I’ll have to go back to the original and update again.<\/ins><\/p>\n\n\n\n

This was done earlier<\/a>, but I have since had to change it to match changes in the incoming data. Some of which is a big mystery. In particular a new line character that is input as the “line separator” Unicode character. It would end up in the clipboard as line feed, so I just started plugging in white space character id’s until one hit the mark. Lucky me.
\n<\/p>\n\n\n\n\n\n\n\n

With that in mind, this is tuned for my specific data, but should be readily taken apart and tweaked to work for others. It’s mostly simple text item delimiter filtering, which is only done a few times.<\/p>\n\n\n\n

Things to note:<\/p>\n\n\n\n

There is a first and\/or last “character” removal subroutine that really does text items, and has no mechanism to check what is being left out of the return string.<\/p>\n\n\n\n

There is a LOT of logging peppered in the main part of the script. Hopefully it (and how to turn it off) is easy to work with. It’s really the only way to nail down what you’re getting and where you’re getting it.<\/p>\n\n\n\n

Last but certainly not least, and the star of our show, there is a (US) state names abbreviation subroutine. Could easily be run in reverse or even set-up with a switch. BTW, you’re welcome. \ud83d\ude00<\/p>\n\n\n\n

PS., You may notice there is a second text abbreviation function. The first one, for state names, is pure AppleScript and use the input as a single object to identify the position from one list that we will pull from the other. The second one, for street address abbreviations, uses sed<\/code> and examines whole lines against the entire list of patterns (with another being matching replacements). You could do the state names with the sed<\/code> routine, but it seems to me it would be slower to do so since at some point you are looking for things guaranteed to not be part of the line you’re feeding it.<\/p>\n\n\n\n

Although I could do two sed<\/code> based routines, one for each line of the address and with their own focused lists. I wouldn’t have to extract the state name doing it that way. Just throw the whole line at it… Hmmm, to be continued?<\/p>\n\n\n\n

-- http:\/\/strawhousepig.net\/\n\n-- Be aware that upper_cased() also removes every \".\" from text that is sent to it.\n\nset _search to \"New submission from \"\n-- String that selected e-mails must contain. Can also be set to \"is\" instead of \"contains\" below.\nset debug to true\n-- Once you get things working set to 'false'.\n\nset theAddresses to \"\"\nset theCount to 0\nset errors to \"\"\n\ntell application \"Mail\"\n\tset _messages to selection as list\n\trepeat with m in _messages\n\t\ttry\n\t\t\tif subject of m contains _search then\n\t\t\t\t-- Can also look at other message properties for the _search string.\n\t\t\t\tset _content to (content of m) as rich text\n\t\t\t\tset my text item delimiters to {\"Name\", \"Email\", \"Address\", \"United States\"}\n\t\t\t\tif debug is true then\n\t\t\t\t\t-- Find which text items contain the data we are after. \n\t\t\t\t\t-- This time we want to label everything because it may be a lot of items, including whitespace items.\n\t\t\t\t\t-- Counting text items after we refine the data here should be a lot less tricky. You can of course do it like this again, too.\n\t\t\t\t\trepeat with t from 1 to count of text items of _content\n\t\t\t\t\t\ttry\n\t\t\t\t\t\t\tlog \"text item \" & t & \" of _content = \" & item t of every text item of _content\n\t\t\t\t\t\tend try\n\t\t\t\t\tend repeat\n\t\t\t\tend if\n\t\t\t\tset _name to text item 2 of _content\n\t\t\t\tset _address to text item 4 of _content\n\t\t\t\tset my text item delimiters to {character id 10, character id 13, character id 8232}\n\t\t\t\t-- 10 = line feed, 13 = carriage return, 8232 = line separator\n\t\t\t\tif debug is true then\n\t\t\t\t\tlog \"_name :\" & text items of _name\n\t\t\t\t\tlog \"_address :\" & text items of _address\n\t\t\t\tend if\n\t\t\t\tset _name to my upper_cased(text item 3 of _name)\n\t\t\t\tset _address1 to my upper_cased(text item 3 of _address)\n\t\t\t\tset _address2 to my upper_cased(text item 4 of _address)\n\t\t\t\tif debug is true then\n\t\t\t\t\tlog \"_name: \" & text items of _name\n\t\t\t\t\tlog \"_address1: \" & text items of _address1\n\t\t\t\t\tlog \"_address2: \" & text items of _address2\n\t\t\t\tend if\n\t\t\t\tset my text item delimiters to {\",\", \"0\", \"1\", \"2\", \"3\", \"4\", \"5\", \"6\", \"7\", \"8\", \"9\"}\n\t\t\t\tset _city to text item 1 of _address2\n\t\t\t\tset _state to text item 2 of _address2 -- This leaves us with a space before and after the text, which we remove later.\n\t\t\t\tset my text item delimiters to {space}\n\t\t\t\tset _zip to last text item of _address2\n\t\t\t\tif debug is true then\n\t\t\t\t\tlog \"_city: \" & _city\n\t\t\t\t\tlog \"_state: \" & _state\n\t\t\t\t\tlog \"_zip: \" & _zip\n\t\t\t\tend if\n\t\t\t\tset _state to my state_abr(my trim_char(_state, \"both\"))\n\t\t\t\tif debug is true then\n\t\t\t\t\tlog \"_state: \" & _state\n\t\t\t\tend if\n\t\t\t\tset theAddresses to theAddresses & _name & return & my address_abr(_address1) & return & _city & \", \" & _state & \" \" & _zip & return & return\n\t\t\t\tset theCount to theCount + 1\n\t\t\telse\n\t\t\t\tset errors to \"Selected e-mail subject does not contain what we are looking for.\"\n\t\t\tend if\n\t\ton error theErr\n\t\t\tset errors to errors & _name & \": \" & theErr & return & return\n\t\tend try\n\tend repeat\n\tif theCount is not 0 then\n\t\t--set the clipboard to theAddresses --swap the comments from this and \"my write2file() below to send to clipboard.\n\t\t--display dialog (theCount as string) & \" addresses ready to paste.\" with icon 1\n\t\tmy write2file(theAddresses)\n\t\t-- Could be turned into a CSV file fairly easy.\n\telse\n\t\tdisplay dialog \"No addresses were extracted from selected e-mails. :(\" with icon 2\n\tend if\n\tif errors is not \"\" then display dialog \"Some funny business was reported:\" & return & return & errors & return & return & \"Run script from ScriptEditor to view more info.\"\nend tell\n\non write2file(_input)\n\tset listFile to ((path to public folder) as text) & \"Parsed addresses \" & (do shell script \"date \\\"+ %Y-%m-%d %H%M\\\"\") & \".txt\"\n\ttell application \"Finder\"\n\t\tset newFile to (open for access file listFile with write permission)\n\t\tset eof newFile to 0\n\t\twrite _input to newFile\n\t\tclose access newFile\n\t\ttell application \"TextEdit\" to open listFile as alias\n\tend tell\nend write2file\n\non trim_char(the_input, first_last_both)\n\tset my text item delimiters to {}\n\tset trim_count to count of text items of the_input\n\tif first_last_both is \"first\" then return text items 2 thru trim_count of the_input as string\n\tif first_last_both is \"last\" then return text items 1 thru (trim_count - 1) of the_input as string\n\tif first_last_both is \"both\" then return text items 2 thru (trim_count - 1) of the_input as string\nend trim_char\n\non upper_cased(_input)\n\treturn (do shell script \"echo \" & quoted form of _input & \" | sed 's\/\\\\.\/\/g' | tr a-z A-Z\")\nend upper_cased\n\non state_abr(_state)\n\t-- States don't have to be listed in all caps as this is case insensitive, but that's how it's sent to this function.\n\tset long_state to {\"ALABAMA\", \"ALASKA\", \"ARIZONA\", \"ARKANSAS\", \"CALIFORNIA\", \"COLORADO\", \"CONNECTICUT\", \"DELAWARE\", \"FLORIDA\", \"GEORGIA\", \"HAWAII\", \"IDAHO\", \"ILLINOIS\", \"INDIANA\", \"IOWA\", \"KANSAS\", \"KENTUCKY\", \"LOUISIANA\", \"MAINE\", \"MARYLAND\", \"MASSACHUSETTS\", \"MICHIGAN\", \"MINNESOTA\", \"MISSISSIPPI\", \"MISSOURI\", \"MONTANA\", \"NEBRASKA\", \"NEVADA\", \"NEW HAMPSHIRE\", \"NEW JERSEY\", \"NEW MEXICO\", \"NEW YORK\", \"NORTH CAROLINA\", \"NORTH DAKOTA\", \"OHIO\", \"OKLAHOMA\", \"OREGON\", \"PENNSYLVANIA\", \"RHODE ISLAND\", \"SOUTH CAROLINA\", \"SOUTH DAKOTA\", \"TENNESSEE\", \"TEXAS\", \"UTAH\", \"VERMONT\", \"VIRGINIA\", \"WASHINGTON\", \"WEST VIRGINIA\", \"WISCONSIN\", \"WYOMING\"}\n\tset short_state to {\"AL\", \"AK\", \"AZ\", \"AR\", \"CA\", \"CO\", \"CT\", \"DE\", \"FL\", \"GA\", \"HI\", \"ID\", \"IL\", \"IN\", \"IA\", \"KS\", \"KY\", \"LA\", \"ME\", \"MD\", \"MA\", \"MI\", \"MN\", \"MS\", \"MO\", \"MT\", \"NE\", \"NV\", \"NH\", \"NJ\", \"NM\", \"NY\", \"NC\", \"ND\", \"OH\", \"OK\", \"OR\", \"PA\", \"RI\", \"SC\", \"SD\", \"TN\", \"TX\", \"UT\", \"VT\", \"VA\", \"WA\", \"WV\", \"WI\", \"WY\"}\n\trepeat with n from 1 to count of long_state\n\t\tif item n of long_state is _state then return item n of short_state\n\tend repeat\n\t-- If someone doesn't know how to spell we need to send the original back.\n\treturn _state\nend state_abr\n\non address_abr(raw_address)\n\t-- WE ARE... We are changing case to upper before sending to this function to avoid style differences. Why?\n\t-- Because `sed` on OS X cannot be used case insensitive (same for NetBSD apparently, and is POSIX compliant?).\n\tset long_address to {\"NORTH\", \"SOUTH\", \"EAST\", \"WEST\", \"NORTHEAST\", \"NORTHWEST\", \"SOUTHEAST\", \"SOUTHWEST\", \"ROAD\", \"STREET\", \"AVENUE\", \"BOULEVARD\", \"PLACE\", \"CIRCLE\", \"DRIVE\", \"LANE\", \"ROUTE\", \"SUITE\", \"APARTMENT\", \"PARKWAY\", \"HIGHWAY\", \"EXPRESSWAY\", \"BYPASS\", \"CAUSEWAY\", \"STRAVENUE\", \"THROUGHWAY\", \"TURNPIKE\", \"VIADUCT\"}\n\tset short_address to {\"N\", \"S\", \"E\", \"W\", \"NE\", \"NW\", \"SE\", \"SW\", \"RD\", \"ST\", \"AVE\", \"BLVD\", \"PL\", \"CIR\", \"DR\", \"LN\", \"RTE\", \"STE\", \"APT\", \"PKWY\", \"HWY\", \"EXPY\", \"BYP\", \"CSWY\", \"STRA\", \"TRWY\", \"TPKE\", \"VIA\"}\n\trepeat with s from 1 to count of long_address\n\t\tset raw_address to (do shell script \"echo \\\"\" & raw_address & \"\\\" | sed 's\/[[:<:]]\" & item s of long_address & \"[[:>:]]\/\" & item s of short_address & \"\/g'\")\n\t\t-- Taking apart the `sed` command: s=replace; \/=Regex delimiter; [[:<:]] & [[:>:]]=PATTERN delineation so we don't have to worry \n\t\t-- about \"NORTHERN\" or \"BROADWAY\", aka word boundary; g=all occurances of PATTERN. Try it out!\n\t\t-- echo BROADWAY ROAD | sed 's\/[[:<:]]ROAD[[:>:]]\/RD\/g'\n\tend repeat\n\treturn raw_address\nend address_abr<\/code><\/pre>\n\n\n\n
The below version works on 10.4 (AppleScript version 1.10), and has not been tested beyond that.<\/p>\n\n\n\n
<\/p>\n\n\n\n
-- http:\/\/strawhousepig.net\/\n\nset _search to \"New submission from \"\n-- String that selected e-mails must contain. Can also be set to \"is\" instead of \"contains\" below.\nset debug to true\n-- If things don't work out as planned, set to true and look at the Replies pane below.\n\nset theAddresses to \"\"\nset theCount to 0\nset errors to \"\"\n\ntell application \"Mail\"\n\tset _messages to selection as list\n\trepeat with m in _messages\n\t\ttry\n\t\t\tif subject of m contains _search then\n\t\t\t\t-- Can also look at other message properties for the _search string.\n\t\t\t\tset _content to content of m\n\t\t\t\tif debug is true then\n\t\t\t\t\t-- Find which lines (paragraphs) contain the data we are after. \n\t\t\t\t\trepeat with p from 1 to count of paragraphs of _content\n\t\t\t\t\t\ttry\n\t\t\t\t\t\t\tlog \"paragraph \" & p & \" = \" & item p of every paragraph of _content\n\t\t\t\t\t\tend try\n\t\t\t\t\tend repeat\n\t\t\t\tend if\n\t\t\t\tset _name to my upper_cased(paragraph 7 of _content)\n\t\t\t\t-- paragraph 13 contains the next set of data I need, however it also contains a Unicode \"line separator\" character.\n\t\t\t\tset _content2 to (do shell script \"echo \" & (paragraph 13 of _content) & \" | grep \\\\n\")\n\t\t\t\t-- This susses out my oddball \"line separator\" character without using AppleScript text item delimiters because,\n\t\t\t\t-- AppleScript versions before 2.0 (OS X 10.5) were not, or not readily, able to work with Unicode characters.\n\t\t\t\t-- We (as in I) are forced to do this since we (as in I) still run OS X 10.4 at times.\n\t\t\t\tif debug is true then\n\t\t\t\t\trepeat with p2 from 1 to count of paragraphs of _content2\n\t\t\t\t\t\ttry\n\t\t\t\t\t\t\tlog \"paragraph2 \" & p2 & \" = \" & paragraph p2 of _content2\n\t\t\t\t\t\tend try\n\t\t\t\t\tend repeat\n\t\t\t\tend if\n\t\t\t\tset _address1 to my upper_cased(paragraph 1 of _content2)\n\t\t\t\tset _address2 to my upper_cased(paragraph 2 of _content2)\n\t\t\t\tif debug is true then\n\t\t\t\t\tlog \"_name: \" & _name\n\t\t\t\t\tlog \"_address1: \" & _address1\n\t\t\t\t\tlog \"_address2: \" & _address2\n\t\t\t\tend if\n\t\t\t\tset my text item delimiters to {\",\"}\n\t\t\t\t-- Not sure if it's just me, but on 10.4 I can't use more than the first delimiter in a list. FML...\n\t\t\t\t-- Isolate city from state and numbers at the end. Hopefully there is always a comma after the city.\n\t\t\t\tset _city to text item 1 of _address2\n\t\t\t\tset _state to my state_abr(words 1 thru ((count of words of text item 2 of _address2) - 1) of text item 2 of _address2)\n\t\t\t\tset _zip to last word of text item 2 of _address2\n\t\t\t\tif debug is true then\n\t\t\t\t\tlog \"_city: \" & _city\n\t\t\t\t\tlog \"_state: \" & _state\n\t\t\t\t\tlog \"_zip: \" & _zip\n\t\t\t\tend if\n\t\t\t\tset theAddresses to theAddresses & _name & return & my address_abr(_address1) & return & _city & \", \" & _state & \" \" & _zip & return & return\n\t\t\t\tset theCount to theCount + 1\n\t\t\telse\n\t\t\t\tset errors to \"Selected e-mail subject does not contain \\\"\" & _search & \"\\\"\"\n\t\t\tend if\n\t\ton error theErr\n\t\t\tset errors to errors & theErr & return & return\n\t\tend try\n\tend repeat\n\tif theCount is not 0 then\n\t\tset the clipboard to theAddresses\n\t\tdisplay dialog (theCount as string) & \" addresses ready to paste.\" with icon 1\n\telse\n\t\tdisplay dialog \"No addresses were extracted from selected e-mails. :(\" with icon 2\n\tend if\n\tif errors is not \"\" then display dialog \"Some funny business was reported:\" & return & return & errors\nend tell\n\non upper_cased(_input)\n\treturn (do shell script \"echo \" & _input & \" | tr a-z A-Z\")\nend upper_cased\n\non state_abr(_state)\n\tset long_state to {\"ALABAMA\", \"ALASKA\", \"ARIZONA\", \"ARKANSAS\", \"CALIFORNIA\", \"COLORADO\", \"CONNECTICUT\", \"DELAWARE\", \"FLORIDA\", \"GEORGIA\", \"HAWAII\", \"IDAHO\", \"ILLINOIS\", \"INDIANA\", \"IOWA\", \"KANSAS\", \"KENTUCKY\", \"LOUISIANA\", \"MAINE\", \"MARYLAND\", \"MASSACHUSETTS\", \"MICHIGAN\", \"MINNESOTA\", \"MISSISSIPPI\", \"MISSOURI\", \"MONTANA\", \"NEBRASKA\", \"NEVADA\", \"NEW HAMPSHIRE\", \"NEW JERSEY\", \"NEW MEXICO\", \"NEW YORK\", \"NORTH CAROLINA\", \"NORTH DAKOTA\", \"OHIO\", \"OKLAHOMA\", \"OREGON\", \"PENNSYLVANIA\", \"RHODE ISLAND\", \"SOUTH CAROLINA\", \"SOUTH DAKOTA\", \"TENNESSEE\", \"TEXAS\", \"UTAH\", \"VERMONT\", \"VIRGINIA\", \"WASHINGTON\", \"WEST VIRGINIA\", \"WISCONSIN\", \"WYOMING\"}\n\tset short_state to {\"AL\", \"AK\", \"AZ\", \"AR\", \"CA\", \"CO\", \"CT\", \"DE\", \"FL\", \"GA\", \"HI\", \"ID\", \"IL\", \"IN\", \"IA\", \"KS\", \"KY\", \"LA\", \"ME\", \"MD\", \"MA\", \"MI\", \"MN\", \"MS\", \"MO\", \"MT\", \"NE\", \"NV\", \"NH\", \"NJ\", \"NM\", \"NY\", \"NC\", \"ND\", \"OH\", \"OK\", \"OR\", \"PA\", \"RI\", \"SC\", \"SD\", \"TN\", \"TX\", \"UT\", \"VT\", \"VA\", \"WA\", \"WV\", \"WI\", \"WY\"}\n\trepeat with n from 1 to count of long_state\n\t\tif item n of long_state is _state then return item n of short_state\n\tend repeat\n\t-- If someone doesn't know how to spell we need to send the original back.\n\treturn _state\nend state_abr\n\non address_abr(raw_address) -- This is some hot garbage right here.\n\t-- WE ARE... We are changing case to upper before sending to this function to avoid style differences. Why?\n\t-- Because `sed` on OS X cannot be used case insensitive (same for NetBSD apparently, and is POSIX compliant?).\n\tset long_address to {\"NORTH\", \"SOUTH\", \"EAST\", \"WEST\", \"NORTHEAST\", \"NORTHWEST\", \"SOUTHEAST\", \"SOUTHWEST\", \"ROAD\", \"STREET\", \"AVENUE\", \"BOULEVARD\", \"PLACE\", \"CIRCLE\", \"DRIVE\", \"LANE\", \"ROUTE\", \"SUITE\", \"APARTMENT\", \"PARKWAY\", \"HIGHWAY\", \"EXPRESSWAY\", \"BYPASS\", \"CAUSEWAY\", \"STRAVENUE\", \"THROUGHWAY\", \"TURNPIKE\", \"VIADUCT\"}\n\tset short_address to {\"N\", \"S\", \"E\", \"W\", \"NE\", \"NW\", \"SE\", \"SW\", \"RD\", \"ST\", \"AVE\", \"BLVD\", \"PL\", \"CIR\", \"DR\", \"LN\", \"RTE\", \"STE\", \"APT\", \"PKWY\", \"HWY\", \"EXPY\", \"BYP\", \"CSWY\", \"STRA\", \"TRWY\", \"TPKE\", \"VIA\"}\n\trepeat with s from 1 to count of long_address\n\t\tset raw_address to (do shell script \"echo \\\"\" & raw_address & \"\\\" | sed 's\/\\\\b\" & item s of long_address & \"\\\\b\/\" & item s of short_address & \"\/g'\")\n\t\t-- Taking apart the `sed` command: s=replace; \/=Regex delimiter; [[:<:]] & [[:>:]]=PATTERN deliniation (so we don't have to worry \n\t\t-- about \"NORTHERN\" or \"BROADWAY\", it only matches whole words); g=all occurances of PATTERN\n\t\t-- sed 's\/[[:<:]]PATTERN[[:>:]]\/REPLACEMENT\/g'\n\tend repeat\n\treturn raw_address\nend address_abr<\/code><\/pre>\n\n\n\n
<\/p>\n","protected":false},"excerpt":{"rendered":"
[UPDATE] Added a version that works on OS X 10.4 to the bottom of this post. Does some different logging and is tighter, so I’ll have to go back to the original and update again. This was done earlier, but I have since had to change it to match changes in the incoming data. Some […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[10,24,28],"class_list":["post-872","post","type-post","status-publish","format-standard","hentry","category-code","tag-applescript-2","tag-macos","tag-os-x"],"_links":{"self":[{"href":"https:\/\/strawhousepig.net\/wordpress\/wp-json\/wp\/v2\/posts\/872","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/strawhousepig.net\/wordpress\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/strawhousepig.net\/wordpress\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/strawhousepig.net\/wordpress\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/strawhousepig.net\/wordpress\/wp-json\/wp\/v2\/comments?post=872"}],"version-history":[{"count":3,"href":"https:\/\/strawhousepig.net\/wordpress\/wp-json\/wp\/v2\/posts\/872\/revisions"}],"predecessor-version":[{"id":1074,"href":"https:\/\/strawhousepig.net\/wordpress\/wp-json\/wp\/v2\/posts\/872\/revisions\/1074"}],"wp:attachment":[{"href":"https:\/\/strawhousepig.net\/wordpress\/wp-json\/wp\/v2\/media?parent=872"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/strawhousepig.net\/wordpress\/wp-json\/wp\/v2\/categories?post=872"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/strawhousepig.net\/wordpress\/wp-json\/wp\/v2\/tags?post=872"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}