[Puddin]

I've been fighting blocked worker threads for a while now. We're running exchange 2003 in a clustered environment with a shared stoage back end. Running BES 4.1.5 (recently upgraded). External sql 2000 database. BES and Exchange are in the same rack - latency not an issue. I estimate about 1000 mail boxes, of those 300 BES users.

We constantly get the worker threads have been blocked for x checks multiple times and then the BES automatically restarts the agents after it finally gives up - sometimes up to 6 times. This causes huge delays in mail processing/delivery.

Can anyone give me some insight? T support has been less than stellar on this issue.

Thanks,

Ken

[Frank Castle]

Basically something is going on that is preventing the BES from doing it's work.

SQL connection dropping? AD accounts disabled and still on the BES? Storage issues?

check your event log and usually right after the hung thread errors you will see an event with some clue if it's user related etc.

[Puddin]

I'm seeing these errors in the event log:


Error: BlackBerry Router:
[SERVICE_RELAY_SESSION:S9870052:00993648] Service transaction not found. SERVICESESSION_TAG=27735747

[Puddin]

I looked on the exchange server and KB902400 is not installed, but I did find it installed on the BES. Would this have the same effect of blocking UDP communication issues?

[fadmin]

This is mostly due to miscommunication between BES and exchange. Maybe exchange was temporarily unaccessible due to any reason (network related or simple reboot). Simple bes service restart usually fixes it if failure cause was fixed.

[Puddin]

Originally Posted by fadmin

This is mostly due to miscommunication between BES and exchange. Maybe exchange was temporarily unaccessible due to any reason (network related or simple reboot). Simple bes service restart usually fixes it if failure cause was fixed.

I would have known if the exchange srv went down and would have appropriateley chalked it up to that. Support and I have been going round and round on this. I realize it's a communications problem, but where do I look to find the root cause?

[cancundan]

Usually when we see lots of chronic hung threads, there is a small % of users that are causing most of the issues. Its a user that has a 12 GB mailbox or 12,000 contacts, or a bunch of recurring meeting that are scheduled until 2011.....with no latency in your environment, that an area to investigate.

Start by locating the users causing the hung threads, then see if its isolated to one mail server, then check some of their settings for those users. Obviously, the more historical data you have the better to see if its always the same users causing them.

I work for Boxtone, and we offer a free system health check that will sift through your logs and isolate the users for you, show you which mail server they are coming from,and how often they are occurring. All we need are some log files and we can produce the reports for you.

send me a private message or come see us at the WES.

[fadmin]

What causes Blocked Worker Thread error messages?

There are multiple reasons for blocked MAPI worker threads.

* The BlackBerry Enterprise Server attempted to contact a BlackBerry device user's mailbox but could not initiate a connection with the Microsoft Exchange Information Store.
* The BlackBerry Enterprise Server attempted to contact the Microsoft Exchange server and is still waiting for a response.
* The Microsoft Exchange server might be offline or another unreachable state.
* The BlackBerry Enterprise Server attempted to contact the Microsoft Exchange server but did not receive a timely response. This could be due to issues such as packet loss on the network or complex calls to the Microsoft Exchange server (e.g. a bulk load on a BlackBerry device user with more than 5000 calendar entries).
* The Microsoft Exchange server may be taxed too heavily and cannot handle the load.

After the cause has been determined, it will probably be necessary to restart the BlackBerry Enterprise Server. After a pool of threads has crashed restarting the BlackBerry Enterprise Server services will not release them, as they are part of the Messaging Application Programming Interface (MAPI) subsystem.

Important: Restarting the BlackBerry Enterprise Server will delay message delivery to BlackBerry devices. For more information, see KB04789.

If only individual threads are hung, restarting the BlackBerry Enterprise Server services will release those threads and reinitialize them without restarting the BlackBerry Enterprise Server. However, since it is difficult to determine whether individual threads have crashed, or an entire pool of threads, it is better to restart the BlackBerry Enterprise Server rather than restarting only the services.

[Puddin]

Originally Posted by Puddin

I looked on the exchange server and KB902400 is not installed, but I did find it installed on the BES. Would this have the same effect of blocking UDP communication issues?

Does anyone think by having this patch on the BES it could affect performance with the threads/agents? From what I've searched, it's bad to have it applied to the exchange server.

[gibson_hg]

This patch causes mail delays to BB, but not because of blocked worker threads. It blocks the BES from receiving notifications of new mail, the BES only sees the new mail when scanning the mailbox 15-20 minutes later.