How to solve munmap_chunk(): invalid pointer error in C++
I have an application in C++ and OpenCV which tries to use the classification model realized with SVMLight in order to add the weight values which can be use for HOG Calssificator under OpenCV.
The problem is that when I run the application, I get this error:
./main Converting Model file...
1%3%4%5%7%8%9%11%12%13%15%16%18%19%20%22%23%24%26%27%28%30%31%32%34%35%36%38%39%40%42%43%45%46%47%49%50%51%53%54%55%57%58%59%61%62%63%65%66%67%69%70%72%73%74%76%77%78%80%81%82%84%85%86%88%89%90%92%93%94%96%97%99%100%Skipped
line
*** glibc detected *** ./main: munmap_chunk(): invalid pointer:
0x09926218 ***
======= Backtrace: ========= /lib/libc.so.6(+0x6c501)[0xb6a3d501]
/lib/libc.so.6(+0x6d77e)[0xb6a3e77e]
/usr/lib/tls/libnvidia-tls.so.260.19.26(+0xa20)[0xb4116a20]
./main(_ZN9__gnu_cxx13new_allocatorIfE10deallocateEPfj+0x11)[0x804f7ab]
./main(_ZNSt12_Vector_baseIfSaIfEE13_M_deallocateEPfj+0x25)[0x804f075]
./main(_ZNSt12_Vector_baseIfSaIfEED2Ev+0x37)[0x804e9f1]
./main(_ZNSt6vectorIfSaIfEED1Ev+0x38)[0x804e46a]
./main[0x804bf65]
/lib/libc.so.6(__libc_start_main+0xe7)[0xb69e7ce7]
./main[0x804bd11]
======= Memory map: ======== 08048000-08052000 r-xp 00000000 08:01
27272122
/home/roccog/HOGImplementation/HOGTrainer/main
08052000-08053000 r--p 00009000 08:01
27272122
/home/roccog/HOGImplementation/HOGTrainer/main
08053000-08054000 rw-p 0000a000 08:01
27272122
/home/roccog/HOGImplementation/HOGTrainer/main
098dd000-0994a000 rw-p 00000000 00:00
0 [heap] b2387000-b238d000
rw-p 00000000 00:00 0
b238d000-b23dc000 r-xp 00000000 08:01
16386856 /usr/lib/libXt.so.6.0.0
b23dc000-b23dd000 r--p 0004e000 08:01
16386856 /usr/lib/libXt.so.6.0.0
b23dd000-b23e0000 rw-p 0004f000 08:01
16386856 /usr/lib/libXt.so.6.0.0
b23e0000-b23e1000 rw-p 00000000 00:00
0 b23e1000-b2422000 r-xp 00000000
08:01 16392261
/usr/lib/libQtXml.so.4.7.0
b2422000-b2423000 r--p 00041000 08:01
16392261 /usr/lib/libQtXml.so.4.7.0
b2423000-b2424000 rw-p 00042000 08:01
16392261 /usr/lib/libQtXml.so.4.7.0
b2424000-b243e000 r-xp 00000000 08:01
16387730 /usr/lib/libv4lconvert.so.0
b243e000-b2440000 r--p 0001a000 08:01
16387730 /usr/lib/libv4lconvert.so.0
b2440000-b2441000 rw-p 0001c000 08:01
16387730 /usr/lib/libv4lconvert.so.0
b2441000-b2491000 rw-p 00000000 00:00
0 b2491000-b2496000 r-xp 00000000
08:01 16387500
/usr/lib/libogg.so.0.7.0
b2496000-b2497000 r--p 00004000 08:01
16387500 /usr/lib/libogg.so.0.7.0
b2497000-b2498000 rw-p 00005000 08:01
16387500 /usr/lib/libogg.so.0.7.0
b2498000-b24eb000 r-xp 00000000 08:01
16387509
/usr/lib/liborc-0.4.so.0.0.0
b24eb000-b24ec000 r--p 00052000 08:01
16387509
/usr/lib/liborc-0.4.so.0.0.0
b24ec000-b24ef000 rw-p 00053000 08:01
16387509
/usr/lib/liborc-0.4.so.0.0.0
b24ef000-b24f0000 rw-p 00000000 00:00
0 b24f0000-b24f4000 r-xp 00000000
08:01 16386828
/usr/lib/libXdmcp.so.6.0.0
b24f4000-b24f5000 r--p 00003000 08:01
16386828 /usr/lib/libXdmcp.so.6.0.0
b24f5000-b24f6000 rw-p 00004000 08:01
16386828 /usr/lib/libXdmcp.so.6.0.0
b24f6000-b24f8000 r-xp 00000000 08:01
16386817 /usr/lib/libXau.so.6.0.0
b24f8000-b24f9000 r--p 00001000 08:01
16386817 /usr/lib/libXau.so.6.0.0
b24f9000-b24fa000 rw-p 00002000 08:01
16386817 /usr/lib/libXau.so.6.0.0
b24fa000-b250f000 r-xp 00000000 08:01
16391993 /usr/lib/libaudio.so.2.4
b250f000-b2510000 r--p 00015000 08:01
16391993 /usr/lib/libaudio.so.2.4
b2510000-b2511000 rw-p 00016000 08:01
16391993 /usr/lib/libaudio.so.2.4
b2511000-b258a000 r-xp 00000000 08:01
16392274 /usr/lib/libQtDBus.so.4.7.0
b258a000-b258b000 r--p 00079000 08:01
16392274 /usr/lib/libQtDBus.so.4.7.0
b258b000-b258c000 rw-p 0007a000 08:01
16392274 /usr/lib/libQtDBus.so.4.7.0
b258c000-b2592000 r-xp 00000000 08:01
16387729 /usr/lib/libv4l2.so.0
b2592000-b2593000 r--p 00005000 08:01
16387729 /usr/lib/libv4l2.so.0
b2593000-b2597000 rw-p 00006000 08:01
16387729 /usr/lib/libv4l2.so.0
b2597000-b2598000 rw-p 00000000 00:00
0 b2598000-b25a3000 r-xp 00000000
08:01 4456631
/lib/libusb-1.0.so.0.0.0
b25a3000-b25a4000 r--p 0000a000 08:01
4456631 /lib/libusb-1.0.so.0.0.0
b25a4000-b25a5000 rw-p 0000b000 08:01
4456631 /lib/libusb-1.0.so.0.0.0
b25a5000-b2630000 r-xp 00000000 08:01
16392999 /usr/lib/libvpx.so.0.9.5
b2630000-b2631000 r--p 0008b000 08:01
16392999 /usr/lib/libvpx.so.0.9.5
b2631000-b2632000 rw-p 0008c000 08:01
16392999 /usr/lib/libvpx.so.0.9.5
b2632000-b263c000 rw-p 00000000 00:00
0 b263c000-b2662000 r-xp 00000000
08:01 16387736
/usr/lib/libvorbis.so.0.4.4
b2662000-b2663000 r--p 00025000 08:01
16387736 /usr/lib/libvorbis.so.0.4.4
b2663000-b2664000 rw-p 00026000 08:01
16387736 /usr/lib/libvorbis.so.0.4.4
b2664000-b27c9000 r-xp 00000000 08:01
16387738
/usr/lib/libvorbisenc.so.2.0.7
b27c9000-b27ca000 ---p 00165000 08:01
16387738
/usr/lib/libvorbisenc.so.2.0.7
b27ca000-b27db000 r--p 00165000 08:01
16387738
/usr/lib/libvorbisenc.so.2.0.7
b27db000-b27dc000 rw-p 00176000 08:01
16387738
/usr/lib/libvorbisenc.so.2.0.7
b27dc000-b27f4000 r-xp 00000000 08:01
16387697
/usr/lib/libtheoradec.so.1.1.4
b27f4000-b27f5000 r--p 00017000 08:01
16387697
/usr/lib/libtheoradec.so.1.1.4
b27f5000-b27f6000 rw-p 00018000 08:01
16387697
/usr/lib/libtheoradec.so.1.1.4
b27f6000-b27f7000 rw-p 00000000 00:00
0 b27f7000-b2839000 r-xp 00000000
08:01 16387699
/usr/lib/libtheoraenc.so.1.1.2
b2839000-b283a000 r--p 00041000 08:01
16387699
/usr/lib/libtheoraenc.so.1.1.2
b283a000-b283b000 rw-p 00042000 08:01
16387699
/usr/lib/libtheoraenc.so.1.1.2
b283b000-b2856000 r-xp 00000000 08:01
16392180
/usr/lib/sse2/libspeex.so.1.5.0
b2856000-b2857000 r--p 0001a000 08:01
16392180
/usr/lib/sse2/libspeex.so.1.5.0
b2857000-b2858000 rw-p 0001b000 08:01
16392180
/usr/lib/sse2/libspeex.so.1.5.0
b2858000-b28f5000 r-xp 00000000 08:01
16392994
/usr/lib/libschroedinger-1.0.so.0.3.0
b28f5000-b28f6000 ---p 0009d000 08:01
16392994
/usr/lib/libschroedinger-1.0.so.0.3.0
b28f6000-b28f7000 r--p 0009d000 08:01
16392994
/usr/lib/libschroedinger-1.0.so.0.3.0
b28f7000-b28f8000 rw-p 0009e000 08:01
16392994
/usr/lib/libschroedinger-1.0.so.0.3.0
b28f8000-b28f9000 rw-p 00000000 00:00
0 b28f9000-b2905000 r-xp 00000000
08:01 16392992
/usr/lib/libgsm.so.1.0.12
b2905000-b2906000 r--p 0000b000 08:01
16392992 /usr/lib/libgsm.so.1.0.12
b2906000-b2907000 rw-p 0000c000 08:01
16392992 /usr/lib/libgsm.so.1.0.12
b2907000-b290c000 r-xp 00000000 08:01
16392996 /usr/lib/libva.so.1.0.1
b290c000-b290d000 r--p 00004000 08:01
16392996 /usr/lib/libva.so.1.0.1
b290d000-b290e000 rw-p 00005000 08:01
16392996 /usr/lib/libva.so.1.0.1
b290e000-b290f000 rw-p 00000000 00:00
0 b290f000-b2933000 r-xp 00000000
08:01 4456520
/lib/libexpat.so.1.5.2
b2933000-b2935000 r--p 00024000 08:01
4456520 /lib/libexpat.so.1.5.2
b2935000-b2936000 rw-p 00026000 08:01
4456520 /lib/libexpat.so.1.5.2
b2936000-b293c000 r-xp 00000000 08:01
16387775
/usr/lib/libxcb-render.so.0.0.0
b293c000-b293d000 r--p 00005000 08:01
16387775
/usr/lib/libxcb-render.so.0.0.0
b293d000-b293e000 rw-p 00006000 08:01
16387775
/usr/lib/libxcb-render.so.0.0.0
b293e000-b2940000 r-xp 00000000 08:01
16387777
/usr/lib/libxcb-shm.so.0.0.0
b2940000-b2941000 r--p 00001000 08:01
16387777
/usr/lib/libxcb-shm.so.0.0.0
b2941000-b2942000 rw-p 00002000 08:01
16387777
/usr/lib/libxcb-shm.so.0.0.0
b2942000-b299e000 r-xp 00000000 08:01
16387543
/usr/lib/libpixman-1.so.0.18.4Aborted
It's strange because it successfully creates the file which I need, but it gives this error anyway.
This is the code which I use:
vector<float> test;
loadSVMfromModelFile("model", &test); // model is the file create with SVM_Light
//loads a file from SVMlight and converts the loaded support vectors to the weight vector.
void loadSVMfromModelFile(const char* filename, vector<float>* svm){
ifstream svinstr (filename);
string line;
float d,g,s,r, b;
int maxidx,numtrain,numsvm, type;
int cur_svidx = 0;
getline(svinstr, line);
line.clear();
svinstr >> type;
if (type != 0){
cout << "Error: Only linear SVM supported" << endl;
return;
}
getline(svinstr, line);
svinstr >> d; //Kernel parameter d...
line.clear();
getline(svinstr, line);
svinstr >>g;
line.clear();
getline(svinstr, line);
svinstr >> s;
line.clear();
getline(svinstr, line);
svinstr >> r;
line.clear();
getline(svinstr, line);
line.clear();
getline(svinstr, line);
svinstr >> maxidx; //highest feature idx
line.clear();
getline(svinstr, line);
svinstr >> numtrain; //num of training vecs
line.clear();
getline(svinstr, line);
svinstr >> numsvm; //num of support vecs
line.clear();
getline(svinstr, line);
svinstr >> b; //offset b;
line.clear();
getline(svinstr, line);
line.clear();
svm->clear();
svm->resize(maxidx+1, 0);
(*svm)[maxidx] = -b;
while(!svinstr.eof())
{
cur_svidx++;
if (cur_svidx%20 ==0)
{
cout << cvRound((double)cur_svidx/(double)numsvm*100) << "%";
flush(cout);
}
getline(svinstr, line);
if (line.size() < 5){
cout << "Skipped line" << endl;
continue;
}
istringstream strstream(line);
float ftemp;
int itemp;
double alpha;
strstream >> alpha;
int lastitemp = -1;
while (!strstream.eof()) {
strstream >> itemp;
if (itemp == lastitemp){
break;
}
lastitemp = itemp;
char x;
strstream >> x;
strstream >>ftemp;
(*svm)[itemp-1] += alpha * ftemp;
}
svinstr.sync();
}
}
Can you help me to improve this utility and remove this weird error, please?
EDIT: This is what valgrind outputs me:
valgrind --leak-check=yes ./main
==27668== Memcheck, a memory error detector
==27668== Copyright (C) 2002-2010, and GNU GPL'd, by Julian Seward et al.
==27668== Using Valgrind-3.6.0.SVN-Debian and LibVEX; rerun with -h for copyright info
==27668== Command: ./main
==27668==
Converting Model file...
==27668== Invalid read of size 4
==27668== at 0x804C6A6: loadSVMfromModelFile(char const*, std::vector<float, std::allocator<float> >*) (main.cpp:182)
==27668== by 0x804BF44: main (main.cpp:67)
==27668== Address 0x9f38844 is 4 bytes before a block of size 15,124 alloc'd
==27668== at 0x4026351: operator new(unsigned int) (vg_replace_malloc.c:255)
==27668== by 0x804F8C5: __gnu_cxx::new_allocator<float>::allocate(unsigned int, void const*) (new_allocator.h:89)
==27668== by 0x804F627: std::_Vector_base<float, std::allocator<float> >::_M_allocate(unsigned int) (in /home/roccog/HOGImplementation/HOGTrainer/main)
==27668== by 0x804F2ED: std::vector<float, std::allocator<float> >::_M_fill_insert(__gnu_cxx::__normal_iterator<float*, std::vector<float, std::allocator<float> > >, unsigned int, float const&) (vector.tcc:414)
==27668== by 0x804EABD: std::vector<float, std::allocator<float> >::insert(__gnu_cxx::__normal_iterator<float*, std::vector<float, std::allocator<float> > >, unsigned int, float const&) (stl_vector.h:859)
==27668== by 0x804E54F: std::vector<float, std::allocator<float> >::resize(unsigned int, float) (stl_vector.h:558)
==27668== by 0x804C4D5: loadSVMfromModelFile(char const*, std::vector<float, std::allocator<float> >*) (main.cpp:152)
==27668== by 0x804BF44: main (main.cpp:67)
==27668==
==27668== Invalid write of size 4
==27668== at 0x804C6C4: loadSVMfromModelFile(char const*, std::vector<float, std::allocator<float> >*) (main.cpp:182)
==27668== by 0x804BF44: main (main.cpp:67)
==27668== Address 0x9f38844 is 4 bytes before a block of size 15,124 alloc'd
==27668== at 0x4026351: operator new(unsigned int) (vg_replace_malloc.c:255)
==27668== by 0x804F8C5: __gnu_cxx::new_allocator<float>::allocate(unsigned int, void const*) (new_allocator.h:89)
==27668== by 0x804F627: std::_Vector_base<float, std::allocator<float> >::_M_allocate(unsigned int) (in /home/roccog/HOGImplementation/HOGTrainer/main)
==27668== by 0x804F2ED: std::vector<float, std::allocator<float> >::_M_fill_insert(__gnu_cxx::__normal_iterator<float*, std::vector<float, std::allocator<float> > >, unsigned int, float const&) (vector.tcc:414)
==27668== by 0x804EABD: std::vector<float, std::allocator<float> >::insert(__gnu_cxx::__normal_iterator<float*, std::vector<float, std::allocator<float> > >, unsigned int, float const&) (stl_vector.h:859)
==27668== by 0x804E54F: std::vector<float, std::allocator<float> >::resize(unsigned int, float) (stl_vector.h:558)
==27668== by 0x804C4D5: loadSVMfromModelFile(char const*, std::vector<float, std::allocator<float> >*) (main.cpp:152)
==27668== by 0x804BF44: main (main.cpp:67)
==27668==
1%3%4%5%7%8%9%11%12%13%15%16%18%19%20%22%23%24%26%27%28%30%31%32%34%35%36%38%39%40%42%43%45%46%47%49%50%51%53%54%55%57%58%59%61%62%63%65%66%67%69%70%72%73%74%76%77%78%80%81%82%84%85%86%88%89%90%92%93%94%96%97%99%100%Skipped line
EDIT 2: As valgrind outputs:
==27668== Invalid read of size 4
==27668== at 0x804C6A6: loadSVMfromModelFile(char const*, std::vector<float, std::allocator<float> >*) (main.cpp:182)
==27668== by 0x804BF44: main (main.cpp:67)
line 182: (*svm)[itemp-1] += alpha * ftemp;
Then:
==27668== by 0x804BF44: main (main.cpp:67)
==27668== Address 0x9f38844 is 4 bytes before a block of size 15,124 alloc'd
==27668== at 0x4026351: operator new(unsigned int) (vg_replace_malloc.c:255)
==27668== by 0x804F8C5: __gnu_cxx::new_allocator<float>::allocate(unsigned int, void const*) (new_allocator.h:89)
line 67:
vector<float> test;
loadSVMfromModelFile("model", &test);
This happens when the pointer passed to free() is not valid or has been modified somehow. I don't really know the details here. The bottom line is that the pointer passed to free() must be the same as returned by malloc(), realloc() and their friends. It's not always easy to spot what the problem is for a novice in their own code or even deeper in a library. In my case, it was a simple case of an undefined (uninitialized) pointer related to branching.
The free() function frees the memory space pointed to by ptr, which must have been returned by a previous call to malloc(), calloc() or realloc(). Otherwise, or if free(ptr) has already been called before, undefined behavior occurs. If ptr is NULL, no operation is performed. GNU 2012-05-10 MALLOC(3)
char *words; // setting this to NULL would have prevented the issue
if (condition) {
words = malloc( 512 );
/* calling free sometime later works here */
free(words)
} else {
/* do not allocate words in this branch */
}
/* free(words); -- error here --
*** glibc detected *** ./bin: munmap_chunk(): invalid pointer: 0xb________ ***/
There are many similar questions here about the related free() and rellocate() functions. Some notable answers providing more details:
*** glibc detected *** free(): invalid next size (normal): 0x0a03c978 ***
*** glibc detected *** sendip: free(): invalid next size (normal): 0x09da25e8 ***
glibc detected, realloc(): invalid pointer
IMHO running everything in a debugger (Valgrind) is not the best option because errors like this are often caused by inept or novice programmers. It's more productive to figure out the issue manually and learn how to avoid it in the future.
The hint is, the output file is created even if you get this error. The automatic deconstruction of vector starts after your code executed. Elements in the vector are deconstructed as well. This is most probably where the error occurs. The way you access the vector is through vector::operator[] with an index read from stream. Try vector::at() instead of vector::operator[]. This won't solve your problem, but will show which assignment to the vector causes error.