<div dir="ltr"><div>Yes, obviously using implementation with poll() is much better practice than select(), but my problem seems to be a little different. The problem that I face is that I run out of file descriptors after certain number of calls to loop(). <br></div><div>Each call consumes few fd's which seem to be closed properly. But in fact these fd's are never reused and I can see them piling in /proc/{pid}/fdinfo. My implementation does not use many fd's, problem is that they stay open and are not reused. Eventually I run out of fd's and face to buffer overflow crash.<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Feb 26, 2020 at 12:59 PM Pino Toscano <<a href="mailto:ptoscano@redhat.com">ptoscano@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Wednesday, 26 February 2020 10:43:27 CET Richard W.M. Jones wrote:<br>
> On Wed, Feb 26, 2020 at 11:21:18AM +0200, Veselin Kozhuharski wrote:<br>
> > Hallo Rich,<br>
> > <br>
> > Here is the fd list and total number just before collectd application<br>
> > crashes. Before that the number of used fd's is constantly increasing. It<br>
> > looks like a fd leak inside libguestfs to me. I am trying to debug the fd<br>
> > handling inside the library.<br>
> > <br>
> > root@localhost:~# less /proc/8829/fdinfo/<br>
> > Display all 1035 possibilities? (y or n)<br>
> > 0 1029 129 161 193 224 256 288 319 350 382 413 445<br>
> > 477 508 54 571 602 634 666 698 729 760 792 823<br>
> > 855 887 918 95 981<br>
> > 1 103 13 162 194 225 257 289 32 351 383 414 446<br>
> > 478 509 540 572 603 635 667 699 73 761 793 824<br>
> > 856 888 919 950 982<br>
> > 10 1030 130 163 195 226 258 29 320 352 384 415 447<br>
> > 479 51 541 573 604 636 668 7 730 762 794 825<br>
> > 857 889 92 951 983<br>
> > 100 1031 131 164 196 227 259 290 321 353 385 416 448<br>
> > 48 510 542 574 605 637 669 70 731 763 795 826<br>
> > 858 89 920 952 984<br>
> > 1000 1032 132 165 197 228 26 291 322 354 386 417 449<br>
> > 480 511 543 575 606 638 67 700 732 764 796 827<br>
> > 859 890 921 953 985<br>
> > 1001 1033 133 166 198 229 260 292 323 355 387 418 45<br>
> > 481 512 544 576 607 639 670 701 733 765 797 828<br>
> > 86 891 922 954 986<br>
> > 1002 1034 134 167 199 23 261 293 324 356 388 419 450<br>
> > 482 513 545 577 608 64 671 702 734 766 798 829<br>
> > 860 892 923 955 987<br>
> > 1003 1035 135 168 2 230 262 294 325 357 389 42 451<br>
> > 483 514 546 578 609 640 672 703 735 767 799 83<br>
> > 861 893 924 956 988<br>
> > 1004 104 136 169 20 231 263 295 326 358 39 420 452<br>
> > 484 515 547 579 61 641 673 704 736 768 8 830<br>
> > 862 894 925 957 989<br>
> > 1005 105 137 17 200 232 264 296 327 359 390 421 453<br>
> > 485 516 548 58 610 642 674 705 737 769 80 831<br>
> > 863 895 926 958 99<br>
> > 1006 106 138 170 201 233 265 297 328 36 391 422 454<br>
> > 486 517 549 580 611 643 675 706 738 77 800 832<br>
> > 864 896 927 959 990<br>
> > 1007 107 139 171 202 234 266 298 329 360 392 423 455<br>
> > 487 518 55 581 612 644 676 707 739 770 801 833<br>
> > 865 897 928 96 991<br>
> > 1008 108 14 172 203 235 267 299 33 361 393 424 456<br>
> > 488 519 550 582 613 645 677 708 74 771 802 834<br>
> > 866 898 929 960 992<br>
> > 1009 109 140 173 204 236 268 3 330 362 394 425 457<br>
> > 489 52 551 583 614 646 678 709 740 772 803 835<br>
> > 867 899 93 961 993<br>
> > 101 11 141 174 205 237 269 30 331 363 395 426 458<br>
> > 49 520 552 584 615 647 679 71 741 773 804 836<br>
> > 868 9 930 962 994<br>
> > 1010 110 142 175 206 238 27 300 332 364 396 427 459<br>
> > 490 521 553 585 616 648 68 710 742 774 805 837<br>
> > 869 90 931 963 995<br>
> > 1011 111 143 176 207 239 270 301 333 365 397 428 46<br>
> > 491 522 554 586 617 649 680 711 743 775 806 838<br>
> > 87 900 932 964 996<br>
> > 1012 112 144 177 208 24 271 302 334 366 398 429 460<br>
> > 492 523 555 587 618 65 681 712 744 776 807 839<br>
> > 870 901 933 965 997<br>
> > 1013 113 145 178 209 240 272 303 335 367 399 43 461<br>
> > 493 524 556 588 619 650 682 713 745 777 808 84<br>
> > 871 902 934 966 998<br>
> > 1014 114 146 179 21 241 273 304 336 368 4 430 462<br>
> > 494 525 557 589 62 651 683 714 746 778 809 840<br>
> > 872 903 935 967 999<br>
> > 1015 115 147 18 210 242 274 305 337 369 40 431 463<br>
> > 495 526 558 59 620 652 684 715 747 779 81 841<br>
> > 873 904 936 968<br>
> > 1016 116 148 180 211 243 275 306 338 37 400 432 464<br>
> > 496 527 559 590 621 653 685 716 748 78 810 842<br>
> > 874 905 937 969<br>
> > 1017 117 149 181 212 244 276 307 339 370 401 433 465<br>
> > 497 528 56 591 622 654 686 717 749 780 811 843<br>
> > 875 906 938 97<br>
> > 1018 118 150 182 213 245 277 308 34 371 402 434 466<br>
> > 498 529 560 592 623 655 687 718 75 781 812 844<br>
> > 876 907 939 970<br>
> > 1019 119 151 183 214 246 278 309 340 372 403 435 467<br>
> > 499 53 561 593 624 656 688 719 750 782 813 845<br>
> > 877 908 94 971<br>
> > 102 12 152 184 215 247 279 31 341 373 404 436 468<br>
> > 5 530 562 594 625 657 689 72 751 783 814 846<br>
> > 878 909 940 972<br>
> > 1020 120 153 185 216 248 28 310 342 374 405 437 469<br>
> > 50 531 563 595 626 658 69 720 752 784 815 847<br>
> > 879 91 941 973<br>
> > 1021 121 154 186 217 249 280 311 343 375 406 438 47<br>
> > 500 532 564 596 627 659 690 721 753 785 816 848<br>
> > 88 910 942 974<br>
> > 1022 122 155 187 218 25 281 312 344 376 407 439 470<br>
> > 501 533 565 597 628 66 691 722 754 786 817 849<br>
> > 880 911 943 975<br>
> > 1023 123 156 188 219 250 282 313 345 377 408 44 471<br>
> > 502 534 566 598 629 660 692 723 755 787 818 85<br>
> > 881 912 944 976<br>
> > 1024 124 157 189 22 251 283 314 346 378 409 440 472<br>
> > 503 535 567 599 63 661 693 724 756 788 819 850<br>
> > 882 913 945 977<br>
> > 1025 125 158 19 220 252 284 315 347 379 41 441 473<br>
> > 504 536 568 6 630 662 694 725 757 789 82 851<br>
> > 883 914 946 978<br>
> > 1026 126 159 190 221 253 285 316 348 38 410 442 474<br>
> > 505 537 569 60 631 663 695 726 758 79 820 852<br>
> > 884 915 947 979<br>
> > 1027 127 16 191 222 254 286 317 349 380 411 443 475<br>
> > 506 538 57 600 632 664 696 727 759 790 821 853<br>
> > 885 916 948 98<br>
> > 1028 128 160 192 223 255 287 318 35 381 412 444 476<br>
> > 507 539 570 601 633 665 697 728 76 791 822 854<br>
> > 886 917 949 980<br>
> > <br>
> > Do you suspect any particular handling inside libguestfs?<br>
> > Thanks!<br>
> <br>
> Yes I guess the select() function here needs to be replaced with poll().<br>
> <br>
> <a href="https://github.com/libguestfs/libguestfs/blob/d9b4e3086e11b18dfc5215a7c4c21ae637be4fe6/lib/command.c#L651" rel="noreferrer" target="_blank">https://github.com/libguestfs/libguestfs/blob/d9b4e3086e11b18dfc5215a7c4c21ae637be4fe6/lib/command.c#L651</a><br>
<br>
Indeed, we ought to. I'll try to get it converted. Also, note for<br>
libguestfs people: select() is used also in the daemon, although that<br>
should not be a problem, since it does not use many libraries, and in<br>
general has few fd's opened at runtime.<br>
<br>
Note that collectd needs similar fixes, as I read in its bug tracker:<br>
<a href="https://github.com/collectd/collectd/pull/3363" rel="noreferrer" target="_blank">https://github.com/collectd/collectd/pull/3363</a><br>
<br>
-- <br>
Pino Toscano</blockquote></div><br clear="all"><br>-- <br><div dir="ltr" class="gmail_signature">
<p style="margin-bottom:0in"><br>
</p>
<table width="481" cellspacing="0" cellpadding="8">
<colgroup><col width="465">
</colgroup><tbody><tr>
<td style="border:medium none;padding:0in" width="465" height="3" bgcolor="#f2f2f2">
<p><font color="#7a8d97"><font face="Arial, sans-serif"><font size="2"><b>Veselin
Kozhuharski</b></font></font></font><font color="#7a8d97"><font face="Arial, sans-serif"><font style="font-size:8pt" size="1"><b>
|</b></font></font></font><font color="#7a8d97"><font face="Arial, sans-serif"><font style="font-size:8pt" size="1"> Software
Engineer</font></font></font></p>
</td>
</tr>
<tr>
<td style="border:medium none;padding:0in" width="465" height="3" bgcolor="#f2f2f2">
<p><font color="#7a8d97"><font face="Arial, sans-serif"><font style="font-size:8pt" size="1">Direct:
+359 2 439 2590 ext. 3912 </font></font></font><font color="#7a8d97"><font face="Arial, sans-serif"><font style="font-size:8pt" size="1"><b>|</b></font></font></font><font color="#7a8d97"><font face="Arial, sans-serif"><font style="font-size:8pt" size="1"> Mobile:
+359 887 412116 | veselin_k</font></font></font><font color="#f05133"><span lang="zxx"><u><a href="mailto:mzabaruk@telco.com" target="_blank"><font face="Arial, sans-serif"><font style="font-size:8pt" size="1">@telco.com</font></font></a></u></span></font><font color="#7a8d97"><font face="Arial, sans-serif"><font style="font-size:8pt" size="1">
</font></font></font>
</p>
</td>
</tr>
<tr>
<td style="border:medium none;padding:0in" width="465" height="3" bgcolor="#f05a23">
<p><font color="#ffffff"><font face="Arial, sans-serif"><font size="2"><b>Telco
Systems | </b></font></font></font><font color="#f05133"><span lang="zxx"><u><a href="http://www.telco.com/" target="_blank"><font face="Arial, sans-serif"><font size="2"><b>www.telco.com</b></font></font></a></u></span></font></p>
</td>
</tr>
<tr>
<td style="border:medium none;padding:0in" width="465" height="3" bgcolor="#f2f2f2">
<p><font color="#7a8d97"><font face="Arial, sans-serif"><font style="font-size:8pt" size="1">Follow
us: </font></font></font><font color="#f05133"><span lang="zxx"><u><a href="http://www.linkedin.com/company/telco-systems" target="_blank"><font face="Arial, sans-serif"><font style="font-size:8pt" size="1">LinkedIn</font></font></a></u></span></font><font color="#999999"><font face="Arial, sans-serif"><font style="font-size:8pt" size="1"> </font></font></font><font color="#ff6600"><font face="Arial, sans-serif"><font style="font-size:8pt" size="1">|</font></font></font><font color="#999999"><font face="Arial, sans-serif"><font style="font-size:8pt" size="1"> </font></font></font><font color="#f05133"><span lang="zxx"><u><a href="http://twitter.com/TelcoSystems" target="_blank"><font face="Arial, sans-serif"><font style="font-size:8pt" size="1">Twitter</font></font></a></u></span></font><font color="#999999"><font face="Arial, sans-serif"><font style="font-size:8pt" size="1"> </font></font></font><font color="#ff6600"><font face="Arial, sans-serif"><font style="font-size:8pt" size="1">|</font></font></font><font color="#999999"><font face="Arial, sans-serif"><font style="font-size:8pt" size="1"> </font></font></font><font color="#f05133"><span lang="zxx"><u><a href="https://www.facebook.com/TelcoSystems" target="_blank"><font face="Arial, sans-serif"><font style="font-size:8pt" size="1">Facebook</font></font></a></u></span></font><font color="#7a8d97"><font face="Arial, sans-serif"><font style="font-size:8pt" size="1"> </font></font></font><font color="#ff6600"><font face="Arial, sans-serif"><font style="font-size:8pt" size="1">|</font></font></font><font color="#7f7f7f"><font face="Arial, sans-serif"><font style="font-size:8pt" size="1"> </font></font></font><font color="#f05133"><span lang="zxx"><u><a href="http://www.youtube.com/TelcoSystems" target="_blank"><font face="Arial, sans-serif"><font style="font-size:8pt" size="1">YouTube</font></font></a></u></span></font><font color="#999999"><font face="Arial, sans-serif"><font style="font-size:8pt" size="1"> </font></font></font><font color="#ff6600"><font face="Arial, sans-serif"><font style="font-size:8pt" size="1">|</font></font></font><font color="#999999"><font face="Arial, sans-serif"><font style="font-size:8pt" size="1"><b> </b></font></font></font><font color="#f05133"><span lang="zxx"><u><a href="http://www.telco.com/blog" target="_blank"><font face="Arial, sans-serif"><font style="font-size:8pt" size="1">Blog</font></font></a></u></span></font><font color="#666666"><font face="Arial, sans-serif"><font style="font-size:8pt" size="1"> </font></font></font><font color="#ff6600"><font face="Arial, sans-serif"><font style="font-size:8pt" size="1">|</font></font></font></p>
</td>
</tr>
</tbody></table>
<p style="margin-bottom:0in"><br>
</p>
<p style="margin-bottom:0in"><br>
</p>
</div>