2i`ddlmZddlmZddlmZddlZddlmZddl m Z m Z m Z m Z mZmZmZddlmZmZdd lmZdd lmZmZdd lmZmZd d lmZmZmZm Z m!Z!m"Z"ddl#m$Z$m%Z%ddl&m'Z'm(Z(ddl)m*Z*m+Z+ddl,m-Z-ddl.m/Z/ddl0m1Z1ddl2m3Z3ddl4m5Z5ddl6m7Z7ddgZ8GddeZ9GddeZ:GddZ;GddZ<Gdd Z=Gd!d"Z>dS)#) annotations)Optional)LiteralN)_legacy_response)BodyOmitQueryHeadersNotGivenomit not_given)maybe_transformasync_maybe_transform)cached_property)SyncAPIResourceAsyncAPIResource)to_streamed_response_wrapper"async_to_streamed_response_wrapper) OutputItemsAsyncOutputItemsOutputItemsWithRawResponseAsyncOutputItemsWithRawResponse OutputItemsWithStreamingResponse%AsyncOutputItemsWithStreamingResponse)SyncCursorPageAsyncCursorPage)run_list_paramsrun_create_params)AsyncPaginatormake_request_options)Metadata)RunListResponse)RunCancelResponse)RunCreateResponse)RunDeleteResponse)RunRetrieveResponseRuns AsyncRunsc eZdZdZed/dZed0dZed1dZeed d d e d d2dZ d d d e dd3d Z eeeed d d e d!d4d*Z d d d e dd5d,Z d d d e dd6d.Zd S)7r),Manage and run evals in the OpenAI platform.returnrc*t|jSr,)r_clientselfs g/root/.openclaw/workspace/.venv-openai/lib/python3.11/site-packages/openai/resources/evals/runs/runs.py output_itemszRuns.output_items(s4<(((RunsWithRawResponsec t|Sa This property can be used as a prefix for any HTTP method call to return the raw response object instead of the parsed content. For more information, see https://www.github.com/openai/openai-python#accessing-raw-response-data-eg-headers )r6r1s r3with_raw_responsezRuns.with_raw_response-s#4(((r5RunsWithStreamingResponsec t|Sz An alternative to `.with_raw_response` that doesn't eagerly read the response body. For more information, see https://www.github.com/openai/openai-python#with_streaming_response )r:r1s r3with_streaming_responsezRuns.with_streaming_response7s)...r5Nmetadataname extra_headers extra_query extra_bodytimeouteval_idstr data_sourcerun_create_params.DataSourcer?Optional[Metadata] | Omitr@ str | OmitrAHeaders | NonerB Query | NonerC Body | NonerD'float | httpx.Timeout | None | NotGivenr&c |std||d|dt|||dtjt ||||t S)  Kicks off a new run for a given evaluation, specifying the data source, and what model configuration to use to test. The datasource will be validated against the schema specified in the config of the evaluation. Args: data_source: Details about the run's data source. metadata: Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard. Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters. name: The name of the run. extra_headers: Send extra headers extra_query: Add additional query parameters to the request extra_body: Add additional JSON properties to the request timeout: Override the client-level default timeout for this request, in seconds 6Expected a non-empty value for `eval_id` but received /evals//runsrGr?r@rArBrCrDbodyoptionscast_to) ValueError_postrr RunCreateParamsr"r& r2rErGr?r@rArBrCrDs r3createz Runs.create@sN caV]aabb bzz $g $ $ $ #. (  "1 )+Q[el&   r5rUrun_idr(c |std||std||d|d|t||||tS)P Get an evaluation run by ID. Args: extra_headers: Send extra headers extra_query: Add additional query parameters to the request extra_body: Add additional JSON properties to the request timeout: Override the client-level default timeout for this request, in seconds rQ5Expected a non-empty value for `run_id` but received rR/runs/rUrXrYrZ_getr"r(r2r_rErArBrCrDs r3retrievez Runs.retrieveys0 caV]aabb b a_U[__`` `yy -g - -V - -(+Q[el(    r5afterlimitorderstatusrArBrCrDrjrk int | OmitrlLiteral['asc', 'desc'] | OmitrmJLiteral['queued', 'in_progress', 'completed', 'canceled', 'failed'] | OmitSyncCursorPage[RunListResponse]c|std||d|dttt |||| t ||||dt jtSa Get a list of runs for an evaluation. Args: after: Identifier for the last run from the previous pagination request. limit: Number of runs to retrieve. order: Sort order for runs by timestamp. Use `asc` for ascending order or `desc` for descending order. Defaults to `asc`. status: Filter runs by status. One of `queued` | `in_progress` | `failed` | `completed` | `canceled`. extra_headers: Send extra headers extra_query: Add additional query parameters to the request extra_body: Add additional JSON properties to the request timeout: Override the client-level default timeout for this request, in seconds rQrRrS)rjrkrlrm)rArBrCrDquery)pagerXmodel)rZ _get_api_listrr$r"rr RunListParams r2rErjrkrlrmrArBrCrDs r3listz Runs.listsJ caV]aabb b!! $g $ $ $0(+'%%!&!&!&"(  $1 "%"   r5r'c |std||std||d|d|t||||tS)G Delete an eval run. Args: extra_headers: Send extra headers extra_query: Add additional query parameters to the request extra_body: Add additional JSON properties to the request timeout: Override the client-level default timeout for this request, in seconds rQrbrRrcrUrdrZ_deleter"r'rgs r3deletez Runs.deletes0 caV]aabb b a_U[__`` `|| -g - -V - -(+Q[el&    r5r%c |std||std||d|d|t||||tS)U Cancel an ongoing evaluation run. Args: extra_headers: Send extra headers extra_query: Add additional query parameters to the request extra_body: Add additional JSON properties to the request timeout: Override the client-level default timeout for this request, in seconds rQrbrRrcrUrdrZr[r"r%rgs r3cancelz Runs.cancels0 caV]aabb b a_U[__`` `zz -g - -V - -(+Q[el&    r5)r-r)r-r6)r-r:rErFrGrHr?rIr@rJrArKrBrLrCrMrDrNr-r&r_rFrErFrArKrBrLrCrMrDrNr-r()rErFrjrJrkrnrlrormrprArKrBrLrCrMrDrNr-rqr_rFrErFrArKrBrLrCrMrDrNr-r'r_rFrErFrArKrBrLrCrMrDrNr-r%__name__ __module__ __qualname____doc__rr4r9r=r rr^rhrzrrr5r3r)r)%sn66)))_))))_)///_//3)-$("&;D7 7 7 7 7 7 @)-$("&;D" " " " " " P! /3]a)-$("&;D: : : : : : F)-$("&;D" " " " " " V)-$("&;D" " " " " " " " r5c eZdZdZed/dZed0dZed1dZeed d d e d d2dZ d d d e dd3d Z eeeed d d e d!d4d*Z d d d e dd5d,Z d d d e dd6d.Zd S)7r*r,r-rc*t|jSr/)rr0r1s r3r4zAsyncRuns.output_items%s  ---r5AsyncRunsWithRawResponsec t|Sr8)rr1s r3r9zAsyncRuns.with_raw_response*s(---r5AsyncRunsWithStreamingResponsec t|Sr<)rr1s r3r=z!AsyncRuns.with_streaming_response4s.d333r5Nr>rErFrGrHr?rIr@rJrArKrBrLrCrMrDrNr&c K|std||d|dt|||dtjd{Vt ||||t d{VS)rPrQrRrSrTNrUrV)rZr[rr r\r"r&r]s r3r^zAsyncRuns.create=sN caV]aabb bZZ $g $ $ $,#. (  "1 )+Q[el&          r5rUr_r(c K|std||std||d|d|t||||td{VS)rarQrbrRrcrUrdNrergs r3rhzAsyncRuns.retrievevs0 caV]aabb b a_U[__`` `YY -g - -V - -(+Q[el(          r5rirjrkrnrlrormrpAAsyncPaginator[RunListResponse, AsyncCursorPage[RunListResponse]]c|std||d|dttt |||| t ||||dt jtSrs)rZrwrr$r"rrrxrys r3rzzAsyncRuns.listsJ caV]aabb b!! $g $ $ $ 1(+'%%!&!&!&"(  $1 "%"   r5r'c K|std||std||d|d|t||||td{VS)r|rQrbrRrcrUrdNr}rgs r3rzAsyncRuns.deletes0 caV]aabb b a_U[__`` `\\ -g - -V - -(+Q[el& "         r5r%c K|std||std||d|d|t||||td{VS)rrQrbrRrcrUrdNrrgs r3rzAsyncRuns.cancels0 caV]aabb b a_U[__`` `ZZ -g - -V - -(+Q[el&           r5)r-r)r-r)r-rrr)rErFrjrJrkrnrlrormrprArKrBrLrCrMrDrNr-rrrrrr5r3r*r*"sn66..._...._.444_4/3)-$("&;D7 7 7 7 7 7 @)-$("&;D" " " " " " P! /3]a)-$("&;D: : : : : : F)-$("&;D" " " " " " V)-$("&;D" " " " " " " " r5c.eZdZd dZed dZdS) r6runsr)r-Nonec@||_tj|j|_tj|j|_tj|j|_tj|j|_tj|j|_dSN)_runsrto_raw_response_wrapperr^rhrzrrr2rs r3__init__zRunsWithRawResponse.__init__ s &> K   )@ M   %< I   '> K   '> K   r5rc4t|jjSr/)rrr4r1s r3r4z RunsWithRawResponse.output_items3s*$**ABBBr5Nrr)r-r)r-rrrrrrr4rr5r3r6r6sN    &CCC_CCCr5r6c.eZdZd dZed dZdS) rrr*r-rc@||_tj|j|_tj|j|_tj|j|_tj|j|_tj|j|_dSr)rrasync_to_raw_response_wrapperr^rhrzrrrs r3rz!AsyncRunsWithRawResponse.__init__:s &D K   )F M   %B I   'D K   'D K   r5rc4t|jjSr/)rrr4r1s r3r4z%AsyncRunsWithRawResponse.output_itemsMs/tz/FGGGr5Nrr*r-r)r-rrrr5r3rr9sN    &HHH_HHHr5rc.eZdZd dZed dZdS) r:rr)r-rc||_t|j|_t|j|_t|j|_t|j|_t|j|_dSr)rrr^rhrzrrrs r3rz"RunsWithStreamingResponse.__init__Ts 2 K   5 M   1 I   3 K   3 K   r5rc4t|jjSr/)rrr4r1s r3r4z&RunsWithStreamingResponse.output_itemsgs0 0GHHHr5Nr)r-rrrr5r3r:r:SsN    &III_IIIr5r:c.eZdZd dZed dZdS) rrr*r-rc||_t|j|_t|j|_t|j|_t|j|_t|j|_dSr)rrr^rhrzrrrs r3rz'AsyncRunsWithStreamingResponse.__init__ns 8 K   ; M   7 I   9 K   9 K   r5rc4t|jjSr/)rrr4r1s r3r4z+AsyncRunsWithStreamingResponse.output_itemss5TZ5LMMMr5Nr)r-rrrr5r3rrmsN    &NNN_NNNr5r)? __future__rtypingrtyping_extensionsrhttpxr_typesrr r r r r r_utilsrr_compatr _resourcerr _responserrr4rrrrrr paginationrr types.evalsrr _base_clientr!r"types.shared_params.metadatar#types.evals.run_list_responser$types.evals.run_cancel_responser%types.evals.run_create_responser&types.evals.run_delete_responser'!types.evals.run_retrieve_responser(__all__r)r*r6rr:rrr5r3rs+#"""""%%%%%% !!!!!!LLLLLLLLLLLLLLLLLL========'''''';;;;;;;;ZZZZZZZZ;:::::::>>>>>>>>AAAAAAAA555555======AAAAAAAAAAAAAAAAAAEEEEEE ; z z z z z ?z z z zz z z z z  z z z zCCCCCCCC4HHHHHHHH4IIIIIIII4NNNNNNNNNNr5