RabbitMQ message loss problems and ensure the reliability of the message – end consumer does not lose the messages and HA (II)

Continue on articles solve the problems and ensure the message is lost RabbitMQ message reliability (a) unfinished part, we talk about high availability, and consumers MQ Server-side issue of how to ensure that messages are not lost?

Part of the contents of the return, we know the news from the production of end to end service, in order to ensure that messages are not lost, what do we have to do?

    Confirm the transmitting side using mode, notice that the Server to inform the sender terminal did not succeed, retransmission is required operation requires additional processing

    Persistent message processing

The above two operations to ensure that messages are not lost to the server, but the non-availability status, hang up if a node, the service is temporarily unavailable, the need to restart, message recovery, the message will be lost because there is disk storage.

This paper begins with the consumer side talk:

RabbitMQ Server messages to consumers how not to lose?

The above article also mentioned, after the consumer access to information, not enough time to complete processing, direct yourself down, because the messages are automatically default to ack, ack at this time RabbitMQ automatic mechanism will notify MQ Server This message has been processed well, this time the message on the lost, is not expected.

So we used a manual ack mechanism to solve this problem, and then notify the MQ Server after the consumer side finished processing logic, so consumers did not finish processing the message does not send ack, to get the message in the case if the consumer did not have time to deal with their own hanging , and this time MQ cluster will automatically perceive, it will re-conscious message to other consumer service instance.

According to the above ideas you need to complete the following two steps:

First: consumer monitor settings manually ack

  1. = channelManager.getListenerChannel(namespace);
  2. this.queue = queue;
  3., false, consumerTag, this);
  4. this.disconnectedCallback.setChannel(channel);

Core code: (queue, false, consumerTag, this); representative of the second parameter set is not automatic ack false

Second: the implementation of the business after the completion of the manual ack

  1. public static void ack(MessageContext context) {
  2. long deliveryTag = context.getEnvelope().getDeliveryTag();
  3. try {
  4. context.getChannel().basicAck(deliveryTag, false);
  5. } catch (IOException e) {
  6. throw new MqAckException ( “ack message error: abnormal connection or distal closed”, context, e);

  7. }
  8. }

Core code: context.getChannel () basicAck (deliveryTag, false);.

Here the package, the needs of the business after the completion of the implementation of their business code, call the method of the object channel ack notice MQServer, that I perform here is over, you can delete.

Note that there is a problem: If you forget to call this context.getChannel () basicAck (deliveryTag, false);.

Or because the code is abnormal, the code has not been implemented, what will happen? Later to find time to write an article about this issue.

RabbitMQ Server messages stored in high availability

When we solved the problems the production side and consumption side, the basic guarantee of the problem message is not lost, but there is a problem is the availability of information, single-node problem, the problem will affect the common node of the message is temporarily unavailable at this time we’ll use the mirror HA cluster model to be guaranteed.

Previous article to solve the problem and ensure the message is lost RabbitMQ message reliability (a) already mentioned, the difference between the message server deployment of three modes, today devoted to introduce the mirror mode.

Mirror mode using at least 3 nodes, two nodes and disks to ensure a memory node, composition:

Set up a mirror, there are some strategies:

    Synchronization to all, generally do not do so, the performance will be greatly affected

    Synchronous machine most N

    Sync only to comply with the specified name nodes

HA command processing strategy template: rabbitmqctl set_policy [-p Vhost] Name Pattern Definition [Priority]

    For each queue in a set of all nodes “rock.wechat” at the beginning of the image, and is set to auto-sync mode

  1. rabbitmqctl set_policy ha-all "^rock.wechat" '{"ha-mode":"all","ha-sync-mode":"automatic"}'
  2. rabbitmqctl set_policy -p rock ha-all "^rock.wechat" '{"ha-mode":"all","ha-sync-mode":"automatic"}'

    Mirror images of each queue is set to two nodes “rock.wechat.” Beginning, and is set to auto-sync mode

  1. rabbitmqctl set_policy -p rock ha-exacly "^rock.wechat" \
  2. '{"ha-mode":"exactly","ha-params":2,"ha-sync-mode":"automatic"}'

    To mirror the queue assigned to each “node.” At the beginning of the specified node

  1. rabbitmqctl set_policy ha-nodes "^nodes\." \
  2. '{"ha-mode":"nodes","ha-params":["[email protected]", "[email protected]"]}'

But: HA mirror queue has a big drawback is: the system throughput will decrease

So mirroring mode, if you want to customize treatment based on specific business rules, business is not so important, it does not matter the message lost scenes, and requirements must be high performance when image can not set.

to sum up

Explain two articles, analyzes about the idea of ​​a highly available messaging middleware issues, there is no specific code in detail, in case of doubt can leave a comment below and I will respond promptly answered, I will gradually improve later in details, to welcome a lot of attention .

Behind plans to update the article as follows:

    What would lead to duplication of spending and how to solve?

    What kind of real business scenarios need to safeguard order and how to ensure that the order of messages?

    How retry failed by between message queue elegant solution to the micro-service interface?

Recommended Reading

RabbitMQ message loss and solve problems to ensure the reliability of the message (a)

IntelliJ IDEA plug-ins necessary to enhance the efficiency of development


If the harvest, please help forward, there will be follow-up articles contributed better, your encouragement is the author of the greatest power!

I welcome the attention of the public number: architect practice, access to exclusive organize learning resources and push dry goods daily.

Leave a Reply