olea
June 19, 2023, 11:15am
1
Hi:
Using v3.7.2:
cd ~/bin/openrefine-3.7.2/
./refine -m 19G
I have this message of Your current configuration is set to use 1400M of memory but I’m not sure if OR is ignoring the 19G parameter or the 1400M is a first memory reserve:
So, I’m not sure if this is a bug or not.
That’s indeed a bug, which should already be fixed in the development version.
opened 02:46AM - 12 Feb 23 UTC
closed 08:25PM - 21 Mar 23 UTC
bug
priority: High
windows
Regression. Using `refine.bat` on Windows, users cannot override the default 14… 00M setting.
This is likely due to the fact that after reading the parameter options for the command line, we also then read the settings from refine.ini and override the command line options.
### To Reproduce
Steps to reproduce the behavior:
1. Start OpenRefine from command line by passing max memory option using `refine /m 2g` or `refine /m 4096M` etc.
### Current Results
REFINE_MEMORY variable is not set finally. Instead `refine.ini` is read lastly and overrides the command line options asked for by the user.
Notice the setting of refine memory to 4G was picked up from the command line option input, but then later down the log output (@echo on) the `refine.ini` is read.
```log
E:\GitHubRepos\OpenRefine>if ""/m"" == ""/m"" goto arg-m
E:\GitHubRepos\OpenRefine>set REFINE_MEMORY=4G
E:\GitHubRepos\OpenRefine>set REFINE_MIN_MEMORY=4G
E:\GitHubRepos\OpenRefine>goto shift2loop
E:\GitHubRepos\OpenRefine>shift
E:\GitHubRepos\OpenRefine>shift
E:\GitHubRepos\OpenRefine>goto loop
E:\GitHubRepos\OpenRefine>if """" == """" goto readIniFile
E:\GitHubRepos\OpenRefine>rem --- Read ini file -----------------------------------------------
E:\GitHubRepos\OpenRefine>if "" == "" set REFINE_INI_PATH=refine.ini
E:\GitHubRepos\OpenRefine>if not exist refine.ini (
echo The system cannot find the file refine.ini
exit /B 1
)
E:\GitHubRepos\OpenRefine>echo Using refine.ini for configuration
Using refine.ini for configuration
E:\GitHubRepos\OpenRefine>for /F "tokens=1,* delims==" %a in (refine.ini) do (set %a=%b )
E:\GitHubRepos\OpenRefine>(set # NOTE: This file is not read if you run the Refine executable directly= )
E:\GitHubRepos\OpenRefine>(set # It is only read of you use the refine shell script or refine.bat= )
E:\GitHubRepos\OpenRefine>(set # FOR DEVELOPERS: you can copy refine.ini and rename it to refine-dev.ini= )
E:\GitHubRepos\OpenRefine>(set # Configurations in refine.ini will be ignored if refine-dev.ini exists= )
E:\GitHubRepos\OpenRefine>(set # refine-dev.ini won't be tracked by Git, so feel free to put your custom configurations in it= )
E:\GitHubRepos\OpenRefine>(set no_proxy="localhost,127.0.0.1" )
E:\GitHubRepos\OpenRefine>(set #REFINE_PORT=3334 )
E:\GitHubRepos\OpenRefine>(set #REFINE_HOST=127.0.0.1 )
E:\GitHubRepos\OpenRefine>(set #REFINE_WEBAPP=main\webapp )
E:\GitHubRepos\OpenRefine>(set # Maximum JVM heap (memory) and max form size allocations= )
E:\GitHubRepos\OpenRefine>(set #REFINE_MAX_FORM_CONTENT_SIZE=1048576 )
E:\GitHubRepos\OpenRefine>(set REFINE_MEMORY=1400M )
E:\GitHubRepos\OpenRefine>(set # Initial (and minimum) size of Java heap= )
E:\GitHubRepos\OpenRefine>(set REFINE_MIN_MEMORY=1400M )
```
### Expected Behavior
Options read from `refine.ini` can be read first.
However, command line options should always be applied last (so they set the final state of options) and a user at the command line has full control as expected without surprises.
### Screenshots
### Versions
- Operating System: Windows 11
- Browser Version: Edge latest
- JRE or JDK Version: JDK 17
- OpenRefine: master branch
### Datasets
<!-- If you are allowed and are OK with making your data public, it would be awesome if you can include or attach the data causing the issue or a URL pointing to where the data is.
If you are concerned about keeping your data private, you can share it selectively by email to developers who work on the issue -->
### Additional context
We should also double check how things work on Linux with `refine` shell script.
I think the regression happened here: https://github.com/OpenRefine/OpenRefine/pull/4220
We are planning a release of 3.7.3 and I think this fix could be included there too.snapshot releases .
1 Like
