Commit ChangeLog (Build 891 to 910) - 4.2, Filters, Watchers, Support Center, Perf

[jstanden]

* [Platform] Did a major restructuring of the core files to reduce memory and increase performance. Devblocks should be lazy-loading (only processing as needed) on more of the uncommon functionality, which speeds up Ajax calls and the most common functionality.

* [Platform] The '/update' controller is no longer part of the core plugin and is now part of the application. This change was long overdue, since the Update process is a bootloader that primes plugins and the database; it has to be able to run before they're in place, or when plugin changes make it impossible to run the app w/o processing the plugins first.

* [Platform] Plugin manifests can now contribute to classloading and URI routing, which improves performance by allowing more lazy-loading. Requests that touch few objects (e.g. Ajax refreshing/pading a view) should be even faster now.

* [Platform] Phased out the use of Zend Framework's Locale/Date/Translation classes. They were too heavy for the little bit we used, and as dependencies they created too many new issues to track down on updates.

* [Config] Attachment names are now properly escaped when downloaded from the 'Attachments' tab in Config (if they include special characters like '&').

* [Support Center] Some visual improvements to logo/footer/styles of the bundled themes.

* [Address Book] Moved the address book pages (Orgs, Addresses, Import) into tabs and made the address book tabs an extension point. This will be used by things like Community Tools which will be able to add a view of Support Center registered users.

* [Address Book] Added the ability to import values from CSV files directly into custom fields for Organizations and Addresses. There is an option to treat blank fields as 'unset' or 'skip'. This functionality is also conscious of the multi checkbox/picklist fields and will delta-set (add new values without changing existing values). New options provided for picklist style fields will automatically modify the custom field to add the new options.

* [Research] Added a new 'Research' top-level page to group together knowledge resources like: Links, Knowledgebase, Fetch & Retrieve. This removes the emphasis from being entirely on the Knowledgebase, which is usually redundant to begin with (copied from blogs/wikis/forums/FAQs). Plugins can add new functionality to the research page.

* [Platform] Moved the Knowledgebase and Fetch & Retrieve functionality into plugins. Installations that didn't make use of this functionality (based on having other processes in place) can now disable them.

* [Research] Removed the 'Links' tab in 'Research' since we're likely not going to duplicate del.icio.us in Cerb when we can just write a plugin to use the dozens of social bookmarking sites out there.

* [F&R] Deprecated and removed the 'Fetch & Retrieve' plugin for the time-being. There are much better ways to pull together the public search resources these days.

* [KB] Combined the 'Browse KB' and 'Search KB' tabs into a single 'Research' tab for 'Knowledgebase'.

* [KB] Added Topic Add/Edit/Delete from the Knowledebase tab in Research. The ACL priv for adding topics (separate from sub-categories) will allow privileged users to maintain the KB now without needing superuser access.

* [Community Tools] Changed the terminology from 'Community Tools' to 'Community Portals'.

* [Research] Added a new research plugin for 'Google Custom Search Engines'. This currently adds a new 'Google Custom Search' tab to Research and a button to the Ticket Reply toolbar for searching when doing support. In the near future this will also include a Support Center plugin to help customers find community content with a single search (broken down by source: forums, blog, docs, roadmap, etc).

* [Google Custom Search] A tab was added to Config for 'Google Custom Search' where you can define your custom search engines to use during support.

* [Localization] Configuration tabs from plugins are now properly translatable.

* [Platform] Added Date/Time criteria in filters. This allows functionality dependent on the day of week (Sat+Sun) or time of day (8AM-5PM). This allows mail routing to be conscious of office hours and SLA schedules. This criteria works great as a sticky+stackable filter that can set ticket-level custom fields (Due+Priority) on all mail in Mail Routing (based on sender) and then defer to subsequent filters to actually move mail to the right group.

* [Platform] Added message content criteria to filters w/ regular expression support. This scans a message line by line looking for matches on keywords or syntax. This was highly requested by people with example use cases like Paypal receipts for various products, where only the message body gives context about what the payment was for (same subject and 'To:'), and each product has a different group.

* [Mail Routing] Revamped the Mail Routing area in Configuration. Now rather than being restricted to routing incoming mail only by To/Cc to group inboxes you can use the full range of filter functionality available to Inbox Filters: sticky, stackable, message headers, date/time, message body content, read/set custom fields. Existing rules will be converted to the new format by the update script.

* [Mail Filtering] Added more filter functionality to pre-parser filters: sticky, date/time criteria, multiple 'To/Cc:' patterns, message body scanning. Since it's not possible to translate from the deprecated 'Group' criteria on Pre-Parser filters to proper e-mail addresses, you should update your filters in Configuration->Mail Filters if the destination group was an important criteria.

* [Pre-Parser Filters] You can now use sender address and organization custom fields as criteria on Pre-Parser filters. This allows functionality like bouncing mail sent to a mailbox (911@*) w/o an SLA.

* [Group Inbox Filters] Inbox Filters can now use the current Date/Time as a criteria.

* [Group Inbox Filters] Inbox Filters can now scan the message content using regular expressions as a criteria.

[jstanden]

* [Mail] [CHD-1114] Ticket links clicked on from lists sorted by the value of a Custom Field return an "Unknown column: cf_X" error

* [Mail Templates] [CHD-1121] #timestamp# not working in 'Open Ticket'

* [Mail] [CHD-1138] One set of requesters dropped when merged tickets have a common requester.

* [Mail Filters] Mail Filters/Routing/Inbox Filters stop counting after 32767 new messages. (JAS: Bumped to signed 3byte for 8.3M counter cap)

* [Address Book] [CHD-1142] Organization 'notes' don't format output with newlines

* [Custom Fields] [CHD-1157] Unable to use 0 (zero) as an option

* [CHD-1094] Added cancel buttons to peek style popup windows:
Address Book -> Bulk Update
Mail -> Send Mail -> E-Mail Templates
Mail -> Open Ticket -> E-Mail Templates * -> Create Template
Home -> Add Worklist
Workspaces (tab) - > edit workspace
Address book -> Organizations -> bulk update
Address book -> Addresses -> bulk update

* [CHD-185] Fixed bullet indenting issue in the installer license step in IE7.

* [CRM/Opportunities] Added a .csv file importer to the Opportunities tab under Activity. It has full custom field support. You can identify duplicate records from the import using any combination of provided fields (e.g. email address PLUS campaign name custom field) which allows you to duplicate information (like email addys) in particular situations. There is also an option to mass assign all imported records to a particular worker (including hits on dupes). This should drastically improve the workflow of working with Opportunities that you import from external sources (mailing list signups, trials, etc.)

* [ACL/Opportunities] Added a permission for importing CRM opportunities.

* [ACL/KB] Added KB permissions for managing topics, categories, and articles.

* [Performance] Major performance optimizations for lists of all types, but especially for ticket lists with thousands (or more) rows. Tested against 100,000s of rows. Custom fields used as displayed columns in all lists are also now far more efficient to retrieve.

* [Performance] When we introduced multi-value custom fields (multi-picklist/multi-checkbox) we had added "LEFT JOIN" and "GROUP BY" statements to most DAO::search() calls (responsible for drawing all lists) to flatten the potential multiple values into a single row per ID, as you only want to see a single row per ticket/task/etc. On long lists (especially tickets) this was highly inefficient -- the LEFT JOIN had to scan every row for hits, and the GROUP BY had to flatten every row regardless of the number of results shown per page. Now all custom fields are joined to their parent object using more efficient subqueries (which the MySQL optimizer runs like an INNER JOIN at the end). Custom fields will still be LEFT JOIN'd (and a slow GROUP BY added) when a custom field is used as a criteria for a list. In such situations it's a very good idea to make your list as specific as possible (open tickets vs. every single ticket, applicable groups, etc).

* [Performance] Removed the ticket<->group JOIN on tickets searches. The group name can be displayed at render-time using the cache (the same way buckets+workers are). This saves a bit of pointless redundancy on returning ticket lists.

* [Performance] Removed 'interesting words' as a default column on ticket search results. This was only displayed as a tooltip on the spam button, and if this functionality is still desired it can be moved to an Ajax request. It was wasting ~255 bytes of memory per row which could be the difference between MySQL sorting on-disk or in-memory.

* [Watchers] E-mail notifications are now handled like mail filters for super flexible workflow. The new notifications can use far more filtering criteria than just a group+bucket: date/time, incoming/outgoing, custom fields, ticket masks, etc. You can now have a single notification rule span any number of groups and buckets. You can also now exclude the 'groups' criteria to receive notifications about any group you're a member of (and you can filter those results by any other criteria). Notifications are now inherently stackable, meaning all matching notifications will send out e-mail; but only one notification will be sent to each unique e-mail address per event. At the moment that means you don't have to be too careful that rules don't overlap, but in the near future we may introduce explicit sticky/stackable options on notification filters when there are more actions than simply forwarding e-mail (e.g. stripping attachments, using a notification template for SMS, etc). This allows all kinds of new notification workflow: receiving 911@* notifications to SMS during the night shift; receiving notifications on a per-ticket/per-sender/per-org basis (especially useful for SLAs or watching an important issue).

* [Platform] Added a new event point (ticket.comment.create) on the creation of new ticket comments. Initially it was needed by the Watchers plugin for notifications, but can now be used by any plugins for whatever purpose (audit log, etc).

* [Watchers] You can now receive watcher e-mail notifications when new comments are added to tickets. You can filter with all the standard ticket-based criteria (e.g. new comments to tickets I own; new comments in Sales on Wednesday).

* [Watchers] Improved the watcher e-mail notifications for new ticket assignments. You can now use all the ticket-based criteria to decide which assignments you want to know about, and what e-mail addresses to notify on a match. This allows new workflow like receiving ticket assignments by SMS when you're out of the office.

* [Watchers] Added a new 'Assigned To' criteria to watcher e-mail notifications. This is especially helpful for situations where you're training, monitoring, or being mentored by a specific team member.

* [CHD-1162] [Print] Bucket is not being included in print out on Display Ticket.

* [Knowledgebase] [CHD-1163] Saving the KB article peek popup (create/edit/delete) now properly refreshes the underlying list of articles from browse or search.

* [Tasks] [CHD-1164] Adding a new task with no title makes it impossible to edit. (Jeff@WGM: Saving will now enforce a default title if blank. For existing tasks the list will now dynamically provide a default title.)

* [Custom Fields] [CHD-1157] Unable to set '0' (zero) as an option on multi-value custom field types (multi-checkbox, multi-picklist)

* [Custom Fields] Added a new 'Worker' custom field type. This can be used for workflow like 'Account Manager' on Organizations and 'Approved by' on Time Tracking entries.

* [Custom Fields] Added a new 'URL' custom field type. This can be used to add links to any supported object that will show up as hyperlinks in lists/peek/etc.

* [Platform] Cleaned up a lot of redundancy in custom field and filter functionality. A lot more code is properly shared now, which will means we'll spend less time the next time we're adding new custom fields or filtering functionality.

* [Support Center] [CHD-473] Support Center - Open ticket - auto fill the reply address, and make the subject field mandatory

* [Support Center] Removed the concept of 'themes' from the Support Center (SC) since it was forcing disparity and redundancy that made it harder to introduce new functionality. In its place there's now the ability to override CSS styles from configuration per SC. We plan to have a bunch of example color schemes on the wiki. Our current focus is on allowing people to match the SC style to their company colors; but we're not focusing on enabling dynamic layout changes to the SC since it introduces unnecessary complexity in front of the upcoming SC plugins. The REST-based Web-API is still the best way to have 100% control over the implementation of your customer-facing interface. The SC is intentionally a cookie-cutter implementation good enough for most people's needs.

[jstanden]

* [Support Center] [CHD-454] [CHD-522] Major Support Center (SC) refactor. Modules now work like plugins, where each SC instance can choose what functionality to enable (Open Ticket + History; KB + Announcements). Consequently, the 'Public Knowledgebase' and 'Contact Form Builder' Community Tools have been deprecated and removed, since you can now create an SC that only provides that particular functionality. This removed a lot of redundancy and will help keep development speedy; and it also has the nice bonus of bringing all SC perks to the those portals (CSS, footer, custom fields, etc.) Existing Knowledgebase and Contact Form Builder community portals will be automatically converted by the update.

* [Support Center] Plug-in modules are now entirely in control of rendering and saving their own configuration. The aggregated configuration of all modules is still shown seamlessly on the 'Helpdesk Setup->Community Portals' tab.

* [Support Center] The Knowledebase functionality of the Support Center has been moved into a module controlled by the Knowledgebase plugin. Now if you disable the KB plugin the references to the KB in the SC will automatically disappear from all instances.

* [Support Center] You can now give a nickname to Community Portal instances. Previously all the instances displayed as the name of the portal, such as "Support Center". This led to situations where you could have a list of ambiguous portals requiring several clicks to find the one you were looking for.

* [Support Center] Modules can now control their own visibility, rather than requiring the messy hack of editing the plugin.xml manifest. This means that plugins can disappear not only by configuration toggles, but also in situations where they would have no content to display. For example, the KB module will hide itself from the module if no categories were set in config. Similarly, the 'Announcements' module will hide itself if no RSS feeds are set. This also allowed several other feature requests, like "Require registration/login to ____".

* [Support Center] You can now reorder the modules in the menu per Support Center instance. This is especially useful with custom modules.

* [Support Center] When visiting a Support Center without providing a module in the path, the default module will now be the first menu item. This is helpful for single module SC's, such as a portal that only shows the knowledgebase. The KB will now be the first thing you see.

* [Support Center] [CHD-421] You can now require registration and login before permitting visitors to open new tickets.

* [Support Center] [CHD-830] You can now require registration and login before permitting visitors to browse the knowledgebase.

* [Support Center] [CHD-858] The 'Home' module now allows you to enter custom HTML content independent of the RSS feeds. This module is also optional, so you don't need to provide a welcome page unless you want to display additional content on your portals.

* [Support Center] The RSS feeds from 'Home' have been moved into a new optional module called 'Announcements'.

* [Support Center/F&R] At the moment, Fetch & Retrieve functionality has been removed from the Support Center. We're currently looking at better ways to accomplish the original global search goal of F&R (such as the 'Google Custom Search' plugin).

* [Support Center] Brought the AbstractView functionality into the Support Center. This means any arbitrary list from any plugin can automatically inherit Ajax-driven paging and sorting functionality; saving a lot of tedious work on creating and delivering new content.

* [Support Center] Implemented search functionality for the Knowledgebase module. Previously this functionality existed in the standalone 'Public KB' but the SC relied on Fetch & Retrieve. The results are shown in the new Ajax-driven views so they're easily paged and sorted.

* [Support Center] The jQuery Javascript DOM/Ajax library is now in the SC scope and available to be shared by all modules and plugins.

* [Support Center] More Look&Feel improvements to the Support Center. More styles have been moved into the global stylesheet and can be overridden from each portal's configuration. The menu will now highlight the active module to give some visual feedback on navigation.

* [Community Portals] Refactored how the Community Portals handle sessions. The new 'UmPortalHelper' static class is now available to all portals and Support Center modules; it provides access to session, fingerprint, and the currently active portal ID. This simplifies a lot of functionality, including Ajax, and removes a lot of redundancy in various objects that each used to read/write the session directly.

* [Support Center] Rather than providing rigid lists for 'Most Popular' or 'Recently Changed', all articles for the portal's selected topics are now displayed on the main page of the Knowledgebase module. They're sortable by most views or updated date. We'll still provide the ability to share RSS feeds with specific sorting from the KB.

* [Community Portals] Community portals can now property pass query string arguments (?param=value) through the proxy. This is required for the new SC ajax functionality and you should update your deployed index.php files for any live portals.

* [Knowledgebase] Fixed a bug in KB's DAO that allowed duplicate rows in results for some articles in multiple categories. This messed up totals in lists.

* [Notifications/Filters/Mail] Any time mail is moved to a new group+bucket it will now behave like it was just delivered to the helpdesk (and notifications will be sent to the proper people). This may create a bit of extra SMTP traffic, but it was a very popular feature request; and it makes sense that 'new mail' notifications should be triggered when messages are manually moved to new groups+buckets. Based on the fact that mail moving into one group can trigger rules that move it elsewhere, this is a pretty complex piece of functionality to get right. We've attempted to only send the single notification at the end of a chain of auto-moves that may be prompted by various filters, but keep an eye out for odd behavior like superfluous notifications (e.g. "middleman" getting a notification in the workflow: Dispatch->Middleman->Support).

* [Platform] Added a new extension point 'cerberusweb.renderer.postbody' to compliment the existing 'cerberusweb.renderer.prebody' point. These extension points allow plugins to respectively render content below or above the normal helpdesk output. For example, the Time Tracking plugin uses the 'prebody' point to render the floating <DIV> for the stopwatch. This was a feature request from 'landre3567' on GitHub.

* [Support Center] The new view functionality has been added to Ticket History Open/Closed lists as well. They're now pageable and sortable.

* [Support Center] Added the ability for logged in customers to search their ticket history. The query will match message content or the left-hand side of ticket masks ("ab" or "abc" will match "ABC-12345-678").

* [Support Center] Added more CSS styles for Ticket History elements, including differentiating between incoming and outgoing messages when displaying a specific ticket. We've had a lot of conflicting requests for adding visualizations to these messages, so now anything should be possible.

* [Support Center] Added a global CSS style for <button> elements in the Support Center. Browsers tend to draw buttons differently and it can make things look pretty horrible if the visitors' desktop styles dictate odd colors for buttons. The default will make buttons that look like the main Cerb4 GUI, but you can override the stylesheet per portal if you have something else in mind.

* [Community Portals] The community portals will now pull their properties from the cache system when possible rather than querying the database constantly. There's a cache per portal so invalidation is efficient.

* [Migration] The 4.2 patch will now enable the new KB plugin (which was always enabled in <=4.1 and shouldn't default to disabled).

* [Knowledgebase/Mail] Improved the Knowledgebase search on the Ticket Reply screen. The content now comes from the Knowledgebase plugin. When searching the search text will auto-highlight for usability so you can run another query immediately if you didn't find what you were looking for.

* [Knowledgebase/Mail] There's a new button in the Knowledgebase section on Ticket Reply to create new knowledgebase articles based on the reply you're writing. This is a long-standing feature request and the 'peek' type windows make it really easy to copy+paste between (and to search quickly to make sure you're not duplicating an article that already exists).

[jstanden]

* [Support Center/Knowledgebase/RSS] [CHD-636] Publish knowledgebase RSS feeds from the Support Center

* [Mail/Tasks] [CHD-1216] The 'Tasks' tab on Display Ticket doesn't show completed tasks. (Jeff@WGM: Now it will show all tasks, but the total on the tab will continue to only count open tasks that need attention).

* [Address Book] [CHD-1187] Organization Notes are not listed in chronological order

* [Feedback Capture] [CHD-1194] If marked text contain international characters the quote is not copied when pressing "capture feedback"

* [Preferences] [CHD-1167] Logging into the Helpdesk from any other URL other than the standard /login will not set the translation or timezone preferences

* [Requester History] [CHD-1176] Ticket list page numbers are reused when going from ticket to ticket

* [Group Inbox Filters] [CHD-1191] Fixed a bug where 'stackable' inbox filters weren't properly moving tickets on subsequent matching filters.

* [Parser] [CHD-1193] Fixed a silly bug in the parser for checking 'postmaster@' or 'mailer-daemon@' senders and always treating them as bounces.

* [Reports] [CHD-1206] A bug with the 'Worker History' report can allow a potential infinite loop. (Thanks to Robert Middleswarth for finding + patching)

* [Reports/Time Tracking] [CHD-1208] Fixed a usability issue with the 'Time Spent Per Worker' report where changing the worker in the dropdown didn't automatically refresh the graph as it does when date ranges are clicked.

* [Mail] [CHD-243] Setting time to "resume a conversation" does not seem to re-open tickets.

* [Login] Without a session the login code will now properly *redirect* to '/login' which allows custom plugins (LDAP, SSO, etc) to handle the login process as expected. Previously this was just changing the current page to 'login' which didn't allow for overloading the '/login' URI or behavior.

* [Platform] Tweaked the ::redirect(DevblocksHttpRequest) code to always return absolute URLs since that's expected by the HTTP "Location:" header. Relative redirect URLs have caused minor issues in some environments.

* [E-mail Notifications] [CHD-1221] Creating a watcher for "ticket replies to me" is not possible

* [E-mail Notifications] [CHD-1201] "Ticket Assignment" event is not affected by criteria.

* [E-mail Notifications] [CHD-1200] Possible to set "Assigned to (blank)" which can be misleading depending on intended behavior.

* [Group Inbox Filters] [CHD-1133] Rather than being redundant, the '-add filter-' popup now uses the same filter editor as Group Config (providing all the new filter functionality). The filter editor will still show the headers for a given ticket as an example when creating a filter from a ticket list, and it will continue to apply the new filter to the rest of the list and refresh it.

* [CRM] Fixed an issue where the Opportunity Tracking (CRM) plugin had a needless dependency on the Feedback Capture plugin.

* [Web-API] [CHD-1227] Warnings in plugins/cerberusweb.webapi/api/App.php

* [Web-API] [CHD-1226] Any /search.xml actions from the Web-API will now also return attributes with the counts for 'page_results' and 'total_results'. This makes it much easier to know when there are additional pages (and how many).

* [IE7] [CHD-1236] In IE7 the Mail Filtering edit window has squished 'value' fields

* [Mail/Pile Sorter] [CHD-1237] Fixed a bug that prevented the pile sorter from working with ticket lists that used 'message content' as a criteria.