mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-07-31 20:32:20 +03:00
2 lines
2.6 KiB
JSON
2 lines
2.6 KiB
JSON
{"author":{"id":"MDQ6VXNlcjc0OTAyNzk=","is_bot":false,"login":"woutercoppens","name":"Wouter Coppens"},"body":"### Problem / Use Case\n\nCurrent local providers are: LM Studio, vLLM, Lemonade Server, Llamafile, Nvidia Triton, Docker Model Runner, XInterference, oobabooga, SD WebUI, ComfyUI\n\n### Proposed Solution\n\nPlease add support for Please add llama.cpp\n\n### Alternatives Considered\n\n_No response_\n\n### Acceptance Criteria\n\nLlama.cpp is added to local providers\n\n### Area\n\nProvider Support\n\n### Related Provider(s)\n\n_No response_\n\n### Additional Context\n\n_No response_\n\n### Expected Test Plan\n\n_No response_","comments":[{"id":"IC_kwDORPf6ys8AAAABBqt2iQ","author":{"login":"hartmark"},"authorAssociation":"CONTRIBUTOR","body":"llama.cpp supports OpenAI style so just add as OpenAI compatible and suffix /v1 on your url","createdAt":"2026-05-08T13:41:01Z","includesCreatedEdit":false,"isMinimized":false,"minimizedReason":"","reactionGroups":[],"url":"https://github.com/diegosouzapw/OmniRoute/issues/1980#issuecomment-4406867593","viewerDidAuthor":false},{"id":"IC_kwDORPf6ys8AAAABB0uF3w","author":{"login":"soyelmismo"},"authorAssociation":"NONE","body":"> llama.cpp supports OpenAI style so just add as OpenAI compatible and suffix /v1 on your url\n\nfor some reason it doesnt work for me. it says isnt finding the models endpoint neither the chat completions endpoint... and obviously i double-triple checked the endpoints and ip and port and everything is correct. even from inside the omniroute's docker container can find the models from the llama server but only the omniroute dashboard is giving me the error","createdAt":"2026-05-11T03:30:51Z","includesCreatedEdit":false,"isMinimized":false,"minimizedReason":"","reactionGroups":[],"url":"https://github.com/diegosouzapw/OmniRoute/issues/1980#issuecomment-4417357279","viewerDidAuthor":false},{"id":"IC_kwDORPf6ys8AAAABB05zyA","author":{"login":"rshinde-asapp"},"authorAssociation":"NONE","body":"do you have OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS=true set in your .env file? Do that and restart, you should be able to setup a openAI style provider.","createdAt":"2026-05-11T04:12:09Z","includesCreatedEdit":false,"isMinimized":false,"minimizedReason":"","reactionGroups":[],"url":"https://github.com/diegosouzapw/OmniRoute/issues/1980#issuecomment-4417549256","viewerDidAuthor":false}],"createdAt":"2026-05-05T19:39:20Z","labels":[{"id":"LA_kwDORPf6ys8AAAACYAlrLw","name":"enhancement","description":"New feature or request","color":"a2eeef"}],"number":1980,"title":"[Feature] Please add llama.cpp to Local Providers","url":"https://github.com/diegosouzapw/OmniRoute/issues/1980"}
|