celery变量共享
eventlet并发读写socket出现Second simultaneous问题

celery配合redis出现redis.exceptions.InvalidResponse Protocol Error

妙音 posted @ 2020年6月21日 09:29 in python with tags Redis Celery , 563 阅读

说明

 
celery+redis实现异步任务,如果涉及到频繁的io调用,可能会出现错误redis.exceptions.InvalidResponse: Protocol Error
 

错误内容

 
redis.exceptions.InvalidResponse: Protocol Error: {, b'\"status\": \"SUCCESS\", \"result\":xxxx \"traceback\": null, \"children\": [], \"date_done\": \"2020-06-04T07:47:38.317661\", \"task_id\": \"fd4e4fa1-f8cc-4083-a287-ac2b7c7d4472\"}'\r\n"
 

查看redis源码

                                                   
#connection.py
def read_response(self):
    response = self._buffer.readline()
    if not response:
        raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)

    byte, response = byte_to_chr(response[0]), response[1:]

    # 从redis源码中可以看出,通信的数据格式不对
    if byte not in ('-', '+', ':', '$', '*'):
        raise InvalidResponse("Protocol Error: %s, %s" %
                              (str(byte), str(response)))
 

原因

                                                   
从redis源码中可以看出,通信的数据格式不对.
具体为什么会格式不会, 猜测可能是celery使用了pipeline, socket重连之后,数据格式发生变化
 
 

解决办法

 
使用evenetlet进行改造,解决socket问题
 
 
                                                   
Avatar_small
Buy Mathworks Matlab 说:
2020年10月13日 18:31

Sewing Machines… [...]any time to read or go to the content or perhaps internet sites we certainly have associated with[...]…

Avatar_small
Erste Hilfe Kurs 说:
2020年10月14日 04:28

I like your post. It is good to see you verbalize from the heart and clarity on this important subject can be easily observed...

Avatar_small
Digital_Zone 说:
2020年10月20日 20:26

An interesting discussion may be worth comment. I do believe that you ought to write much more about this topic, it might certainly be a taboo subject but usually folks are too few to communicate on such topics. Yet another. Cheers Ramen en deuren

Avatar_small
Digital_Zone 说:
2020年10月20日 20:35

I impressed, I need to say. Really not often do I encounter a blog that each educative and entertaining, and let me tell you, you have hit the nail on the head. Your concept is outstanding; the problem is something that not sufficient persons are speaking intelligently about. I’m very comfortable that I stumbled across this in my seek for one thing referring to this. Vloerisolaties

Avatar_small
Digital_Zone 说:
2020年10月20日 20:37

This is my first time i visit here. I found so many interesting stuff in your blog especially its discussion. From the tons of comments on your articles, I guess I am not the only one having all the enjoyment here keep up the good work PVC ramen

Avatar_small
Digital_Zone 说:
2020年10月20日 20:41

Took me time to read all the comments, but I really enjoyed the article. It proved to be Very helpful to me and I am sure to all the commenters here! It’s always nice when you can not only be informed, but also entertained! Chape

Avatar_small
Digital_Zone 说:
2020年10月20日 20:43

This is a great inspiring article.I am pretty much pleased with your good work.You put really very helpful information. Keep it up. Keep blogging. Looking to reading your next post. Aluminium ramen

Avatar_small
Digital_Zone 说:
2020年10月20日 20:45

You make so many great points here that I read your article a couple of times. Your views are in accordance with my own for the most part. This is great content for your readers. Chapewerken

Avatar_small
Digital_Zone 说:
2020年10月20日 20:46

I admire this article for the well-researched content and excellent wording. I got so involved in this material that I couldn’t stop reading. I am impressed with your work and skill. Thank you so much. Noten kopen

Avatar_small
Digital_Zone 说:
2020年10月20日 20:49

What a fantabulous post this has been. Never seen this kind of useful post. I am grateful to you and expect more number of posts like these. Thank you very much. PVC ramen en deuren

Avatar_small
Digital_Zone 说:
2020年10月20日 20:52

I admit, I have not been on this web page in a long time... however it was another joy to see It is such an important topic and ignored by so many, even professionals. I thank you to help making people more aware of possible issues. Vloerisolatie

Avatar_small
Digital_Zone 说:
2020年10月20日 20:54

very interesting post.this is my first time visit here.i found so mmany interesting stuff in your blog especially its discussion..thanks for the post! Ramen deuren

Avatar_small
Digital_Zone 说:
2020年10月20日 20:57

I can see that you are an expert at your field! I am launching a website soon, and your information will be very useful for me.. Thanks for all your help and wishing you all the success in your business. Notenshop online

Avatar_small
Digital_Zone 说:
2020年10月20日 20:59

This is such a great resource that you are providing and you give it away for free. I love seeing blog that understand the value. Im glad to have found this post as its such an interesting one! I am always on the lookout for quality posts and articles so i suppose im lucky to have found this! I hope you will be adding more in the future... Airco

Avatar_small
Digital_Zone 说:
2020年10月20日 21:02

Great job for publishing such a beneficial web site. Your web log isn’t only useful but it is additionally really creative too. There tend to be not many people who can certainly write not so simple posts that artistically. Continue the nice writing Webdesign

Avatar_small
Digital_Zone 说:
2020年10月20日 21:04

I have read a few of the articles on your website now, and I really like your style of blogging. I added it to my favorites blog site list and will be checking back soon. Please check out my site as well and let me know what you think. Noten

Avatar_small
We Buy Houses in Mil 说:
2020年10月20日 21:04

Positive site, where did u come up with the information on this posting?I have read a few of the articles on your website now, and I really like your style. Thanks a million and please keep up the effective work.

Avatar_small
Digital_Zone 说:
2020年10月20日 21:06

Nice to be visiting your blog again, it has been months for me. Well this article that i’ve been waited for so long. I need this article to complete my assignment in the college, and it has same topic with your article. Thanks, great share. Ramen in PVC

Avatar_small
Digital_Zone 说:
2020年10月20日 21:08

I found this is an informative and interesting post so i think so it is very useful and knowledgeable. I would like to thank you for the efforts you have made in writing this article. Webdesign

Avatar_small
Digital_Zone 说:
2020年10月20日 21:14

You make so many great points here that I read your article a couple of times. Your views are in accordance with my own for the most part. This is great content for your readers. Airconditioning

Avatar_small
Digital_Zone 说:
2020年10月20日 21:17

Only aspire to mention ones content can be as incredible. This clarity with your post is superb and that i may think you’re a guru for this issue. High-quality along with your concur permit me to to seize your current give to keep modified by using approaching blog post. Thanks a lot hundreds of along with you should go on the pleasurable get the job done. Webdesign

Avatar_small
Digital_Zone 说:
2020年10月20日 21:19

An fascinating discussion is value comment. I think that it is best to write extra on this matter, it won’t be a taboo topic however generally people are not enough to talk on such topics. To the next. Cheers Airco installatie

Avatar_small
Digital_Zone 说:
2020年10月20日 21:21

I am unable to read articles online very often, but I’m glad I did today. This is very well written and your points are well-expressed. Please, don’t ever stop writing. Cashewnoten

Avatar_small
Digital_Zone 说:
2020年10月20日 21:23

That is the excellent mindset, nonetheless is just not help to make every sence whatsoever preaching about that mather. Virtually any method many thanks in addition to i had endeavor to promote your own article in to delicius nevertheless it is apparently a dilemma using your information sites can you please recheck the idea. thanks once more. Pur spuiten

Avatar_small
Digital_Zone 说:
2020年10月20日 21:25

I think this is an informative post and it is very beneficial and knowledgeable. Therefore, I would like to thank you for the endeavors that you have made in writing this article. All the content is absolutely well-researched. Thanks... Airco installateur

Avatar_small
Digital_Zone 说:
2020年10月20日 21:27

You have performed a great job on this article. It’s very precise and highly qualitative. You have even managed to make it readable and easy to read. You have some real writing talent. Thank you so much. Isolatiechape

Avatar_small
Digital_Zone 说:
2020年10月20日 21:29

I am incapable of reading articles online very often, but I’m happy I did today. It is very well written, and your points are well-expressed. I request you warmly, please, don’t ever stop writing. Reiki

Avatar_small
Digital_Zone 说:
2020年10月20日 21:32

Wow, What an Outstanding post. I found this too much informatics. It is what I was seeking for. I would like to recommend you that please keep sharing such type of info.If possible, Thanks. Granola

Avatar_small
Digital_Zone 说:
2020年10月20日 21:34

Fabulous post, you have denoted out some fantastic points, I likewise think this s a very wonderful website. I will visit again for more quality contents and also, recommend this site to all. Thanks. Pur isolatie

Avatar_small
buy benzylpiperazine 说:
2020年10月26日 04:38

Love to read it,Waiting For More new Update and I Already Read your Recent Post its Great Thanks. buy benzylpiperazine with Paypal online

Avatar_small
Albert Strickland 说:
2020年11月07日 18:54

I read your blog frequently, and I just thought I’d say keep up the fantastic work! It is one of the most outstanding blogs in my opinion. Yogurt smoothie


登录 *


loading captcha image...
(输入验证码)
or Ctrl+Enter