celery配合redis出现redis.exceptions.InvalidResponse Protocol Error
实用工具状态机transitions

eventlet并发读写socket出现Second simultaneous问题

妙音 posted @ 2020年7月08日 10:50 in python with tags eventlet , 3691 阅读
 

描述

 
celery+eventlet实现任务调用时出现RuntimeError: Second simultaneous read on fileno 14 detected.  Unless you really know what you're doing, make sure that only one greenthread can read any particular socket
 
 

模拟错误

 
网上找的一段代码，模拟出同样问题
 
def main():
    import eventlet
    httplib2 = eventlet.import_patched('httplib2')
    shared_resource = httplib2.Http()

    def get_url():
        resp, content = shared_resource.request("http://eventlet.net")
        return content

    p = eventlet.GreenPile()
    p.spawn(get_url)
    p.spawn(get_url)
    results = list(p)
    assert results[0] == results[1]


if __name__ == "__main__":
    main()
 

错误内容

 
Traceback (most recent call last):
  File "/root/miniconda3/envs/defect/lib/python3.7/site-packages/eventlet/hubs/poll.py", line 111, in wait
    listener.cb(fileno)
  File "/root/miniconda3/envs/defect/lib/python3.7/site-packages/eventlet/greenthread.py", line 221, in main
    result = function(*args, **kwargs)
  File "test.py", line 12, in get_url
    resp, content = shared_resource.request("http://eventlet.net")
  File "/root/miniconda3/envs/defect/lib/python3.7/site-packages/httplib2/__init__.py", line 1994, in request
    cachekey,
  File "/root/miniconda3/envs/defect/lib/python3.7/site-packages/httplib2/__init__.py", line 1651, in _request
    conn, request_uri, method, body, headers
  File "/root/miniconda3/envs/defect/lib/python3.7/site-packages/httplib2/__init__.py", line 1558, in _conn_request
    conn.request(method, request_uri, body, headers)
  File "/root/miniconda3/envs/defect/lib/python3.7/http/client.py", line 1252, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/root/miniconda3/envs/defect/lib/python3.7/http/client.py", line 1298, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/root/miniconda3/envs/defect/lib/python3.7/http/client.py", line 1247, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/root/miniconda3/envs/defect/lib/python3.7/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/root/miniconda3/envs/defect/lib/python3.7/http/client.py", line 987, in send
    self.sock.sendall(data)
  File "/root/miniconda3/envs/defect/lib/python3.7/site-packages/eventlet/greenio/base.py", line 403, in sendall
    tail = self.send(data, flags)
  File "/root/miniconda3/envs/defect/lib/python3.7/site-packages/eventlet/greenio/base.py", line 397, in send
    return self._send_loop(self.fd.send, data, flags)
  File "/root/miniconda3/envs/defect/lib/python3.7/site-packages/eventlet/greenio/base.py", line 392, in _send_loop
    timeout_exc=_timeout_exc)
  File "/root/miniconda3/envs/defect/lib/python3.7/site-packages/eventlet/greenio/base.py", line 210, in _trampoline
    mark_as_closed=self._mark_as_closed)
  File "/root/miniconda3/envs/defect/lib/python3.7/site-packages/eventlet/hubs/__init__.py", line 157, in trampoline
    listener = hub.add(hub.WRITE, fileno, current.switch, current.throw, mark_as_closed)
  File "/root/miniconda3/envs/defect/lib/python3.7/site-packages/eventlet/hubs/epolls.py", line 22, in add
    listener = hub.BaseHub.add(self, evtype, fileno, cb, tb, mac)
  File "/root/miniconda3/envs/defect/lib/python3.7/site-packages/eventlet/hubs/hub.py", line 181, in add
    evtype, fileno, evtype, cb, bucket[fileno]))
RuntimeError: Second simultaneous write on fileno 5 detected.  Unless you really know what you're doing, make sure that only one greenthread can write any particular socket.  Consider using a pools.Pool. If you do know what you're doing and want to disable this error, call eventlet.debug.hub_prevent_multiple_readers(False) - MY THREAD=<built-in method switch of GreenThread object at 0x7f25df161680>; THAT THREAD=FdListener('write', 5, <built-in method switch of GreenThread object at 0x7f25df161470>, <built-in method throw of GreenThread object at 0x7f25df161470>)
 

 解决方法

 
猜测可能是没绿化完整，改成如下，果然就正常了。
 
def main():
    import eventlet
#     httplib2 = eventlet.import_patched('httplib2')
    import http2
    eventlet.monkey_patch()
    shared_resource = httplib2.Http()

    def get_url():
        resp, content = shared_resource.request("http://eventlet.net")
        return content

    p = eventlet.GreenPile()
    p.spawn(get_url)
    p.spawn(get_url)
    results = list(p)
    assert results[0] == results[1]


if __name__ == "__main__":
    main()
 

结论

  • 局部import的模块，可能使用了未被绿化的模块，导出eventlet出现问题
  • 延迟加载的模块配合eventlet也会出现同样问题
 

 

Small_feed 评论 (47)
1
[回复]
Avatar_small
PTC Creo 7.0.1.0 + H 说:
2020年10月13日 18:31

I’ve been surfing online greater than three hours these days, but I by no means found any interesting article like yours. It¡¦s beautiful value sufficient for me. In my view, if all website owners and bloggers made excellent content as you probably did, the net will likely be a lot more helpful than ever before.

2
[回复]
Avatar_small
wheat straws 说:
2020年10月23日 01:36

The Market Leader In Wheat Straw Production Globally wheat straws

3
[回复]
Avatar_small
Digital_Zone 说:
2020年10月24日 03:15 hey there and thank you for your information ¡V I¡¦ve certainly picked up something new from right here. I did however expertise some technical points using this site, since I experienced to reload the website a lot of times previous to I could get it to load properly. I had been wondering if your web host is OK? Not that I am complaining, but sluggish loading instances times will sometimes affect your placement in google and can damage your high quality score if advertising and marketing with Adwords. Anyway I am adding this RSS to my e-mail and could look out for much more of your respective intriguing content. Ensure that you update this again very soon.. Ramen en deuren
4
[回复]
Avatar_small
Digital_Zone 说:
2020年10月24日 03:20

I’m going to read this. I’ll be sure to come back. thanks for sharing. and also This article gives the light in which we can observe the reality. this is very nice one and gives indepth information. thanks for this nice article... Vloerisolaties

5
[回复]
Avatar_small
Digital_Zone 说:
2020年10月24日 03:22

Thanks for a very interesting blog. What else may I get that kind of info written in such a perfect approach? I’ve a undertaking that I am simply now operating on, and I have been at the look out for such info. PVC ramen

6
[回复]
Avatar_small
Digital_Zone 说:
2020年10月24日 03:24

Thanks for taking the time to discuss this, I feel strongly about it and love learning more on this topic. If possible, as you gain expertise, would you mind updating your blog with extra information? It is extremely helpful for me. Chape

7
[回复]
Avatar_small
Digital_Zone 说:
2020年10月24日 03:25

The the next occasion I just read a blog, I really hope that it doesnt disappoint me around this place. Come on, man, I know it was my choice to read, but I really thought youd have some thing fascinating to convey. All I hear is actually a number of whining about something that you could fix if you werent too busy seeking attention. Aluminium ramen

8
[回复]
Avatar_small
Digital_Zone 说:
2020年10月24日 03:30

Wow, What a Excellent post. I really found this to much informatics. It is what i was searching for.I would like to suggest you that please keep sharing such type of info.Thanks Chapewerken

9
[回复]
Avatar_small
Digital_Zone 说:
2020年10月24日 03:32

Positive site, where did u come up with the information on this posting?I have read a few of the articles on your website now, and I really like your style. Thanks a million and please keep up the effective work. Noten kopen

10
[回复]
Avatar_small
Digital_Zone 说:
2020年10月24日 03:33

Hey what a brilliant post I have come across and believe me I have been searching out for this similar kind of post for past a week and hardly came across this. Thank you very much and will look for more postings from you. PVC ramen en deuren

11
[回复]
Avatar_small
Digital_Zone 说:
2020年10月24日 03:35

I can see that you are an expert at your field! I am launching a website soon, and your information will be very useful for me.. Thanks for all your help and wishing you all the success in your business. Vloerisolatie

12
[回复]
Avatar_small
Digital_Zone 说:
2020年10月24日 03:37

Very interesting blog. Alot of blogs I see these days don't really provide anything that I'm interested in, but I'm most definately interested in this one. Just thought that I would post and let you know. Ramen deuren

13
[回复]
Avatar_small
Digital_Zone 说:
2020年10月24日 05:09

Hello, I have browsed most of your posts. This post is probably where I got the most useful information for my research. Thanks for posting, maybe we can see more on this. Are you aware of any other websites on this subject. Notenshop online

14
[回复]
Avatar_small
Digital_Zone 说:
2020年10月24日 05:13

I haven’t any word to appreciate this post.....Really i am impressed from this post....the person who create this post it was a great human..thanks for shared this with us. Airco

15
[回复]
Avatar_small
Digital_Zone 说:
2020年10月24日 05:16

Great job for publishing such a beneficial web site. Your web log isn’t only useful but it is additionally really creative too. There tend to be not many people who can certainly write not so simple posts that artistically. Continue the nice writing Webdesign

16
[回复]
Avatar_small
Digital_Zone 说:
2020年10月24日 05:18

Positive site, where did u come up with the information on this posting? I'm pleased I discovered it though, ill be checking back soon to find out what additional posts you include. Noten

17
[回复]
Avatar_small
Digital_Zone 说:
2020年10月24日 05:20

This is a smart blog. I mean it. You have so much knowledge about this issue, and so much passion. You also know how to make people rally behind it, obviously from the responses. Ramen in PVC

18
[回复]
Avatar_small
Digital_Zone 说:
2020年10月24日 05:21

This is a good post. This post gives truly quality information. I’m definitely going to look into it. Really very useful tips are provided here. Thank you so much. Keep up the good works. Webdesign

19
[回复]
Avatar_small
Digital_Zone 说:
2020年10月24日 05:23

Wow, cool post. I’d like to write like this too – taking time and real hard work to make a great article… but I put things off too much and never seem to get started. Thanks though. Airconditioning

20
[回复]
Avatar_small
Digital_Zone 说:
2020年10月24日 05:25

Your content is nothing short of brilliant in many ways. I think this is engaging and eye-opening material. Thank you so much for caring about your content and your readers. Webdesign

21
[回复]
Avatar_small
Digital_Zone 说:
2020年10月24日 05:27

Nice post. I was checking constantly this blog and I am impressed! Extremely helpful information specially the last part I care for such info a lot. I was seeking this particular information for a very long time. Thank you and good luck. Airco installatie

22
[回复]
Avatar_small
Digital_Zone 说:
2020年10月24日 05:28

Good website! I truly love how it is easy on my eyes it is. I am wondering how I might be notified whenever a new post has been made. I have subscribed to your RSS which may do the trick? Have a great day! Cashewnoten

23
[回复]
Avatar_small
Digital_Zone 说:
2020年10月24日 05:30

That is the excellent mindset, nonetheless is just not help to make every sence whatsoever preaching about that mather. Virtually any method many thanks in addition to i had endeavor to promote your own article in to delicius nevertheless it is apparently a dilemma using your information sites can you please recheck the idea. thanks once more. Pur spuiten

24
[回复]
Avatar_small
Digital_Zone 说:
2020年10月24日 05:32

You delivered such an impressive piece to read, giving every subject enlightenment for us to gain information. Thanks for sharing such information with us due to which my several concepts have been cleared. Airco installateur

25
[回复]
Avatar_small
Digital_Zone 说:
2020年10月24日 05:34

Wow, excellent post. I'd like to draft like this too - taking time and real hard work to make a great article. This post has encouraged me to write some posts that I am going to write soon. Isolatiechape

26
[回复]
Avatar_small
Digital_Zone 说:
2020年10月24日 05:36

Excellent post. I was always checking this blog, and I’m impressed! Extremely useful info specially the last part, I care for such information a lot. I was exploring this particular info for a long time. Thanks to this blog my exploration has ended. Reiki

27
[回复]
Avatar_small
Digital_Zone 说:
2020年10月24日 05:39

Thanks for an interesting blog. What else may I get that sort of info written in such a perfect approach? I have an undertaking that I am just now operating on, and I have been on the lookout for such info. Granola

28
[回复]
Avatar_small
Digital_Zone 说:
2020年10月24日 05:42

You have performed a great job on this article. It’s very precise and highly qualitative. You have even managed to make it readable and easy to read. You have some real writing talent. Thank you so much. Pur isolatie

29
[回复]
Avatar_small
woocommerce etsy dro 说:
2020年10月30日 01:57

This type of message always inspiring and I prefer to read quality content, so happy to find good place to many here in the post, the writing is just great, thanks for the post. woocommerce etsy dropshipping

30
[回复]
Avatar_small
visit this site 说:
2020年12月15日 21:46

Remarkable article, it is particularly useful! I quietly began in this, and I'm becoming more acquainted with it better! Delights, keep doing more and extra impressive!

31
[回复]
Avatar_small
alam 说:
2020年12月19日 16:15

It is the kind of information I have been trying to find. Thank you for writing this information. It has proved utmost beneficial for me. 인싸포커

32
[回复]
Avatar_small
section 80g 说:
2020年12月19日 21:41

It is truly a well-researched content and excellent wording. I got so engaged in this material that I couldn’t wait reading. I am impressed with your work and skill. Thanks.

33
[回复]
Avatar_small
alam 说:
2020年12月21日 15:59

This is a smart blog. I mean it. You have so much knowledge about this issue, and so much passion. You also know how to make people rally behind it, obviously from the responses. custom velcro patches

34
[回复]
Avatar_small
ac repair in Delhi 说:
2020年12月22日 01:45

Your work is truly appreciated round the clock and the globe. It is incredibly a comprehensive and helpful blog.

35
[回复]
Avatar_small
토토사이트 说:
2020年12月23日 15:21

You have done a amazing job with you website

36
[回复]
Avatar_small
Pak24tv 说:
2020年12月26日 13:21

I am happy to find your distinguished way of writing the post. Now you make it easy for me to understand and implement the concept. Thank you for the post. fb status how to stop hair loss how to make money online how to win friends and influence people

37
[回复]
Avatar_small
먹튀 说:
2020年12月26日 23:13

Thanks for taking the time to discuss this, I feel strongly about it and love learning more on this topic. If possible, as you gain expertise, would you mind updating your blog with extra information? It is extremely helpful for me.

38
[回复]
Avatar_small
먹튀 说:
2020年12月27日 00:22

I admire what you have done here. I like the part where you say you are doing this to give back but I would assume by all the comments that this is working for you as well.

39
[回复]
Avatar_small
t shirts with saying 说:
2021年1月05日 23:53

Great Information sharing .. I am very happy to read this article .. thanks for giving us go through info.Fantastic nice. I appreciate this post.

40
[回复]
Avatar_small
pet waste removal Co 说:
2021年1月06日 01:46

Hello! I just want to give a huge thumbs up for the great info you have here on this blog. I will be coming back aimed at your website for additional soon.

41
[回复]
Avatar_small
alam 说:
2021年1月07日 18:03

howdy, your websites are really good. I appreciate your work. <a href="https://www.tolove777.com">안전놀이터</a>

42
[回复]
Avatar_small
funny t shirts for w 说:
2021年1月08日 23:27

Pretty good post. I just stumbled upon your blog and wanted to say that I have really enjoyed reading your blog posts. Any way I'll be subscribing to your feed and I hope you post again soon. Big thanks for the useful info.

43
[回复]
Avatar_small
buy youtube subscrib 说:
2021年1月10日 03:16

Please continue this great work and I look forward to more of your awesome blog posts.

44
[回复]
Avatar_small
where notary lakelan 说:
2021年1月11日 01:29

It was a very good post indeed. I thoroughly enjoyed reading it in my lunch time. Will surely come and visit this blog more often. Thanks for sharing.

45
[回复]
Avatar_small
Voyance telephone es 说:
2021年1月11日 04:32

If your looking for Online Illinois license plate sticker renewals then you have need to come to the right place.We offer the fastest Illinois license plate sticker renewals in the state.

46
[回复]
Avatar_small
Imamia Quran Academy 说:
2023年8月07日 06:03

Nice blog. I enjoyed reading your articles. This is truly a great read for me. I am from Imamia Quran Academy. Shia Quran student is free to choose the timings of classes, lecturers and tutors according to their feasibility and availability to learn Quran. Shia Online Learning more efficiently saves time and money spent over conveyance and physical going to any conventional Shia Institute.

47
[回复]
Avatar_small
pai manis 说:
2023年12月17日 22:11

<div style="display:none">
<a title="mpo slot" href="https://rsudcimacan.cianjurkab.go.id/assets/fe/css/images/mpo-gacor/" rel="Dofollow"><strong>mpo slot</strong></a>
<a title="slot thailand" href="https://sdabk.sumbarprov.go.id/assets/thailand/" rel="Dofollow"><strong>slot thailand</strong></a>
<a title="slot gacor" href="https://sdabk.sumbarprov.go.id/images/2023/12/link-gacor/" rel="Dofollow"><strong>slot gacor</strong></a>
<a title="pulsa 5000" href="https://krs.unpand.ac.id/arsip/pulsa-5000/" rel="Dofollow"><strong>pulsa 5000</strong></a>
<a title="live casino" href="https://sdabk.sumbarprov.go.id/assets/online/" rel="Dofollow"><strong>live casino</strong></a>
<a title="slot gacor maxwin" href="https://sdm.potensi-utama.ac.id/kuis/link-gacor/" rel="Dofollow"><strong>slot gacor maxwin</strong></a>
</div>


登录 *


loading captcha image...
(输入验证码)
or Ctrl+Enter